Jul 15 13:29:15.526620 ip-10-0-141-134 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 13:29:15.944803 ip-10-0-141-134 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 13:29:15.944803 ip-10-0-141-134 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 13:29:15.944803 ip-10-0-141-134 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 13:29:15.944803 ip-10-0-141-134 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 13:29:15.944803 ip-10-0-141-134 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 13:29:15.945692 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.945599    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 13:29:15.950668 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950649    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:29:15.950668 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950667    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950672    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950675    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950679    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950681    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950684    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950687    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950690    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950693    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950696    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950698    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950701    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950704    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950710    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950713    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950716    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950719    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950721    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950724    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950726    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:29:15.950732 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950729    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950732    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950734    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950737    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950740    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950743    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950746    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950749    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950751    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950754    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950756    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950758    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950761    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950764    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950767    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950771    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950775    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950778    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950781    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:29:15.951191 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950784    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950787    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950790    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950792    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950795    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950797    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950800    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950803    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950805    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950808    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950810    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950815    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950818    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950821    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950824    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950828    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950830    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950833    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950835    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950838    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:29:15.951662 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950841    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950844    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950846    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950849    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950852    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950855    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950858    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950861    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950866    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950869    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950871    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950874    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950877    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950879    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950882    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950884    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950887    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950892    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950895    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950897    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:29:15.952303 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950900    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950903    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950906    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950908    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950911    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950914    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950917    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950919    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950922    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950924    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950927    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950929    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950932    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950934    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950937    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950939    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950942    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950945    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:29:15.953060 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.950947    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:29:15.953755 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953734    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:29:15.953755 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953756    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953760    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953763    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953766    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953769    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953772    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953776    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953779    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953782    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953785    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953788    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953791    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953794    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953797    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953800    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953803    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953805    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953808    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953811    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953814    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:29:15.953846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953816    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953819    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953822    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953825    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953831    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953834    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953837    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953839    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953842    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953844    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953847    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953850    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953852    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953855    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953858    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953863    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953867    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953870    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953873    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953875    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:29:15.954354 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953877    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953880    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953882    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953885    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953888    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953890    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953892    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953895    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953897    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953900    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953902    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953905    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953907    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953910    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953913    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953916    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953919    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953922    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953925    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953927    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:29:15.954877 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953930    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953932    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953935    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953937    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953940    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953943    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953945    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953949    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953951    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953954    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953956    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953959    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953961    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953963    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953966    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953968    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953971    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953973    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953976    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953978    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:29:15.955372 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953981    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953983    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953987    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953989    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953992    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.953997    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954001    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954004    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954007    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954010    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954013    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954015    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954018    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954020    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954025    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954033    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954036    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954039    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954473    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:29:15.955855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954479    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954482    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954485    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954488    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954491    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954493    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954496    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954499    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954501    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954504    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954507    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954509    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954512    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954514    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954517    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954520    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954523    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954526    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954529    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:29:15.956328 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954531    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954534    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954537    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954539    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954542    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954544    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954547    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954550    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954553    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954555    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954558    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954560    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954562    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954565    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954567    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954570    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954572    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954575    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954577    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:29:15.956783 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954582    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954585    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954587    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954590    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954592    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954595    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954597    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954600    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954602    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954604    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954609    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954613    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954617    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954620    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954623    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954626    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954629    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954632    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954635    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:29:15.957241 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954637    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954641    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954644    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954647    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954650    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954652    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954655    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954657    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954660    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954662    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954664    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954668    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954671    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954673    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954676    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954678    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954681    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954684    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954687    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:29:15.957709 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954689    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954692    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954694    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954697    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954699    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954702    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954705    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954708    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954710    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954713    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954715    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954718    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954720    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954722    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954725    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954728    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954730    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954733    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954735    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954738    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:29:15.958155 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954740    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.954743    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954827    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954835    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954841    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954846    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954850    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954853    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954857    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954862    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954865    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954868    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954872    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954875    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954878    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954882    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954885    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954888    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954893    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954897    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954902    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954904    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954908    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954911    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 13:29:15.958648 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954914    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954918    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954921    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954923    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954927    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954930    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954933    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954935    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954939    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954941    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954946    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954949    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954952    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954955    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954958    2577 flags.go:64] FLAG: --enable-server="true"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954961    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954966    2577 flags.go:64] FLAG: --event-burst="100"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954968    2577 flags.go:64] FLAG: --event-qps="50"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954971    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954974    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954978    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954982    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954985    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954988    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954991    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 15 13:29:15.959218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954994    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.954998    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955001    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955004    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955007    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955010    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955013    2577 flags.go:64] FLAG: --feature-gates=""
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955017    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955020    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955023    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955026    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955029    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955032    2577 flags.go:64] FLAG: --help="false"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955035    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-141-134.ec2.internal"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955038    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955041    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955044    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955047    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955051    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955054    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955057    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955060    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955063    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 13:29:15.959848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955066    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955069    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955072    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955075    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955078    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955081    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955084    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955087    2577 flags.go:64] FLAG: --lock-file=""
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955089    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955093    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955096    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955102    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955105    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955108    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955111    2577 flags.go:64] FLAG: --logging-format="text"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955114    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955118    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955120    2577 flags.go:64] FLAG: --manifest-url=""
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955124    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955128    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955131    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955135    2577 flags.go:64] FLAG: --max-pods="110"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955138    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955141    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955144    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 13:29:15.960467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955147    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955150    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955153    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955156    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955163    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955166    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955169    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955173    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955175    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955181    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955184    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955187    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955190    2577 flags.go:64] FLAG: --port="10250"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955193    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955196    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e77ee46a6d392e02"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955199    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955205    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955208    2577 flags.go:64] FLAG: --register-node="true"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955211    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955215    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955218    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955221    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955224    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955228    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955231    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 13:29:15.961050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955234    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955236    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955239    2577 flags.go:64] FLAG: --runonce="false"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955242    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955245    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955248    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955251    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955254    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955257    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955276    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955283    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955286    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955289    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955292    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955295    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955298    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955301    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955304    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955309    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955311    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955314    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955319    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955322    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955326    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955329    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 13:29:15.961661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955332    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955335    2577 flags.go:64] FLAG: --v="2"
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955339    2577 flags.go:64] FLAG: --version="false"
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955344    2577 flags.go:64] FLAG: --vmodule=""
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955348    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.955351    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955487    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955491    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955499    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955503    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955505    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955508    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955511    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955514    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955516    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955519    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955522    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955526    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955529    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955532    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955535    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:29:15.962290 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955537    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955540    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955542    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955545    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955548    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955550    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955553    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955556    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955558    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955564    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955567    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955571    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955574    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955576    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955579    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955582    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955585    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955587    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955590    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:29:15.962798 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955592    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955595    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955598    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955600    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955602    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955605    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955607    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955609    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955612    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955616    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955619    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955621    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955624    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955626    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955629    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955631    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955634    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955636    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955639    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955641    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:29:15.963288 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955644    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955646    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955650    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955652    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955655    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955657    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955660    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955662    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955665    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955668    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955671    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955673    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955676    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955678    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955682    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955685    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955688    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955691    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955694    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:29:15.963808 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955696    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955699    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955703    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955705    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955708    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955710    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955713    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955715    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955718    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955720    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955723    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955725    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955728    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955730    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955734    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955739    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955741    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955744    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955747    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:29:15.964380 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955750    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955752    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955755    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955757    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955761    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955764    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.955766    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.956440    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.963170    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.963187    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.963202    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.963208    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963306    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963312    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:29:15.964846 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963315    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963318    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963321    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963324    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963326    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963329    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963332    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963334    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963337    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963339    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963342    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963345    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963348    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963350    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963353    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963355    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963358    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963360    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963362    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963365    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:29:15.965198 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963367    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963372    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963376    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963378    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963381    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963384    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963387    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963401    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963404    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963407    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963410    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963413    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963416    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963420    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963424    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963427    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963431    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963434    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963438    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:29:15.965689 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963440    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963443    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963446    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963449    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963452    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963454    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963457    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963460    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963463    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963466    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963468    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963471    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963473    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963476    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963478    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963481    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963484    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963486    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963489    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963491    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:29:15.966149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963495    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963499    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963502    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963504    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963507    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963510    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963512    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963515    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963517    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963520    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963522    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963525    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963527    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963530    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963532    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963535    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963538    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963541    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963544    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963546    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:29:15.966653 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963549    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963551    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963554    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963556    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963558    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963561    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963563    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963565    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963568    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963571    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963573    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963576    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963579    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963582    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963585    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963588    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963590    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:29:15.967180 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963592    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.963597    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963710    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963714    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963717    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963720    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963723    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963725    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963728    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963730    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963732    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963735    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963737    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963740    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963742    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 13:29:15.967611 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963745    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963748    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963750    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963752    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963755    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963757    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963760    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963762    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963765    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963767    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963769    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963772    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963775    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963777    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963780    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963783    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963786    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963788    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963791    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963794    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 13:29:15.967974 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963796    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963798    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963801    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963803    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963806    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963808    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963811    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963815    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963819    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963822    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963826    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963829    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963832    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963835    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963838    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963840    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963843    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963845    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963848    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 13:29:15.968469 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963850    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963853    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963855    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963858    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963861    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963864    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963867    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963869    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963872    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963875    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963878    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963880    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963883    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963885    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963888    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963890    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963893    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963896    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963900    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 13:29:15.968914 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963903    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963905    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963908    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963911    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963913    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963916    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963919    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963922    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963924    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963927    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963930    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963933    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963935    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963938    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963940    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963943    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963945    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963948    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963951    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963954    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 13:29:15.969378 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963956    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963959    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963962    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963964    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963967    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963970    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963972    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:15.963974    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.963979    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.964104    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.966110    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.969426    2577 server.go:1019] "Starting client certificate rotation"
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.969524    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 13:29:15.969842 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.969567    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 13:29:15.973966 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.973950    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 13:29:15.975747 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.975725    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 13:29:15.986835 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.986809    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 15 13:29:15.988814 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.988796    2577 log.go:25] "Validated CRI v1 image API"
Jul 15 13:29:15.989568 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.989552    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 13:29:15.991889 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.991869    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ec838dc7-37e4-4f25-aebc-60ffe307eebc:/dev/nvme0n1p3 f476486f-1b05-4a18-a83c-c1ce0f03247a:/dev/nvme0n1p4]
Jul 15 13:29:15.991959 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.991888    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 13:29:15.998041 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.997927    2577 manager.go:216] Machine: {Timestamp:2026-07-15 13:29:15.99660796 +0000 UTC m=+0.339543050 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100453 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20ba2fa27e4d77508293f4075805e8 SystemUUID:ec20ba2f-a27e-4d77-5082-93f4075805e8 BootID:c85e091a-2425-46b4-89af-86ddf3c20957 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:21:38:05:a7:05 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:21:38:05:a7:05 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:32:ae:c5:5f:2c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 13:29:15.998041 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.998033    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 13:29:15.998160 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:15.998122    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 13:29:16.000950 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.000931    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 13:29:16.001855 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.001823    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 13:29:16.002005 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.001859    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-134.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 13:29:16.003053 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.003041    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 13:29:16.003087 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.003057    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 13:29:16.003087 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.003065    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 13:29:16.003740 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.003729    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 13:29:16.004803 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.004793    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 13:29:16.007753 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.007741    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 13:29:16.007883 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.007873    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 13:29:16.007918 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.007903    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 13:29:16.007918 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.007916    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 13:29:16.007971 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.007940    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 13:29:16.007971 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.007954    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 15 13:29:16.007971 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.007964    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 13:29:16.008389 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.008377    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 13:29:16.008461 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.008396    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 13:29:16.010438 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.010419    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 13:29:16.013043 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013030    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 13:29:16.013091 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013061    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 13:29:16.013894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013883    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 13:29:16.013927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013900    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 13:29:16.013927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013905    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 13:29:16.013927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013911    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 13:29:16.013927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013919    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 13:29:16.013927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013924    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 13:29:16.014061 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013943    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 13:29:16.014061 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013948    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 13:29:16.014061 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013956    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 13:29:16.014061 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013961    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 13:29:16.014061 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013971    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 13:29:16.014061 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.013978    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 13:29:16.014844 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.014832    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 13:29:16.014885 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.014846    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 13:29:16.018382 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.018367    2577 server.go:1268] "Started kubelet"
Jul 15 13:29:16.018543 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.018511    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 13:29:16.018584 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.018570    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 13:29:16.018891 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.018789    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 13:29:16.019497 ip-10-0-141-134 systemd[1]: Started Kubernetes Kubelet.
Jul 15 13:29:16.019693 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.019677    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 13:29:16.023462 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.022924    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-134.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 13:29:16.023462 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.023370    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 13:29:16.023753 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.023727    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 13:29:16.025068 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.025042    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 13:29:16.025388 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.025370    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.028338    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.028344    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.028589    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029029    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029040    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029047    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.028183    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-134.ec2.internal.18c279a5f1607da9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-134.ec2.internal,UID:ip-10-0-141-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-134.ec2.internal,},FirstTimestamp:2026-07-15 13:29:16.018335145 +0000 UTC m=+0.361270216,LastTimestamp:2026-07-15 13:29:16.018335145 +0000 UTC m=+0.361270216,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-134.ec2.internal,}"
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029187    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029195    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029450    2577 factory.go:55] Registering systemd factory
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029465    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029690    2577 factory.go:153] Registering CRI-O factory
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029706    2577 factory.go:223] Registration of the crio container factory successfully
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029760    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029784    2577 factory.go:103] Registering Raw factory
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.029799    2577 manager.go:1196] Started watching for new ooms in manager
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.030397    2577 manager.go:349] Starting recovery of all containers
Jul 15 13:29:16.030894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.030533    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bkn6t"
Jul 15 13:29:16.036565 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.036535    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 13:29:16.036664 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.036568    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 13:29:16.038773 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.038464    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bkn6t"
Jul 15 13:29:16.040457 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.040439    2577 manager.go:354] Recovery completed
Jul 15 13:29:16.044883 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.044869    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:29:16.046715 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.046692    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:29:16.046771 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.046723    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:29:16.046771 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.046733    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:29:16.047693 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.047682    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 13:29:16.047693 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.047692    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 13:29:16.047770 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.047698    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:29:16.047770 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.047707    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 13:29:16.048549 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.048534    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:29:16.048628 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.048564    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:29:16.048628 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.048581    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:29:16.049006 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.048992    2577 policy_none.go:49] "None policy: Start"
Jul 15 13:29:16.049071 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.049009    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 13:29:16.049071 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.049020    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 13:29:16.049421 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.049356    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-134.ec2.internal.18c279a5f31171dc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-134.ec2.internal,UID:ip-10-0-141-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-134.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-134.ec2.internal,},FirstTimestamp:2026-07-15 13:29:16.046709212 +0000 UTC m=+0.389644286,LastTimestamp:2026-07-15 13:29:16.046709212 +0000 UTC m=+0.389644286,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-134.ec2.internal,}"
Jul 15 13:29:16.050030 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.050018    2577 policy_none.go:47] "Start"
Jul 15 13:29:16.089017 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.088996    2577 manager.go:337] "Starting Device Plugin manager"
Jul 15 13:29:16.089157 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.089036    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 13:29:16.089157 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.089045    2577 server.go:87] "Starting device plugin registration server"
Jul 15 13:29:16.089401 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.089387    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 13:29:16.089482 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.089404    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 13:29:16.089529 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.089503    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 13:29:16.089606 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.089593    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 13:29:16.089670 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.089606    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 13:29:16.090153 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.090125    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 13:29:16.090256 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.090191    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.138953 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.138916    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 13:29:16.140385 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.140368    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 13:29:16.140451 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.140406    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 13:29:16.140451 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.140432    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 13:29:16.140521 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.140477    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 13:29:16.142525 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.142507    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:29:16.190565 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.190522    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:29:16.191531 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.191515    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:29:16.191588 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.191548    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:29:16.191588 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.191559    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:29:16.191588 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.191584    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.199208 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.199140    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.199208 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.199172    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-134.ec2.internal\": node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.213302 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.213278    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.240649 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.240622    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"]
Jul 15 13:29:16.240738 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.240690    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:29:16.242307 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.242293    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:29:16.242369 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.242322    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:29:16.242369 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.242331    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:29:16.243606 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.243591    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:29:16.243766 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.243752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.243807 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.243782    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:29:16.244409 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.244394    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:29:16.244481 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.244421    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:29:16.244481 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.244438    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:29:16.244481 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.244397    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:29:16.244581 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.244496    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:29:16.244581 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.244506    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:29:16.245691 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.245674    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.245783 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.245706    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 13:29:16.246496 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.246477    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 13:29:16.246577 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.246513    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 13:29:16.246577 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.246528    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeHasSufficientPID"
Jul 15 13:29:16.268094 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.268070    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-134.ec2.internal\" not found" node="ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.272515 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.272499    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-134.ec2.internal\" not found" node="ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.314141 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.314109    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.330281 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.330230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae2238e507d170fbad58cb43abb1c04a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"ae2238e507d170fbad58cb43abb1c04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.330422 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.330291    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a01437e717367c58578eaaf88d68b2a5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-134.ec2.internal\" (UID: \"a01437e717367c58578eaaf88d68b2a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.330422 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.330316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae2238e507d170fbad58cb43abb1c04a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"ae2238e507d170fbad58cb43abb1c04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.414707 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.414655    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.431082 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.431055    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae2238e507d170fbad58cb43abb1c04a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"ae2238e507d170fbad58cb43abb1c04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.431154 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.431085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae2238e507d170fbad58cb43abb1c04a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"ae2238e507d170fbad58cb43abb1c04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.431154 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.431136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a01437e717367c58578eaaf88d68b2a5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-134.ec2.internal\" (UID: \"a01437e717367c58578eaaf88d68b2a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.431242 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.431155    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae2238e507d170fbad58cb43abb1c04a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"ae2238e507d170fbad58cb43abb1c04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.431242 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.431196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae2238e507d170fbad58cb43abb1c04a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal\" (UID: \"ae2238e507d170fbad58cb43abb1c04a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.431354 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.431254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a01437e717367c58578eaaf88d68b2a5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-134.ec2.internal\" (UID: \"a01437e717367c58578eaaf88d68b2a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.515553 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.515458    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.570959 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.570928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.574986 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.574747    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:16.616558 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.616527    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.717067 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.717017    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.819979 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.819899    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.922514 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:16.922466    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:16.934373 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.934350    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:29:16.967071 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.967036    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 13:29:16.967721 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.967207    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 13:29:16.967721 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:16.967239    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 13:29:17.022743 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:17.022693    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:17.041550 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:17.041497    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 13:24:16 +0000 UTC" deadline="2028-01-03 11:14:00.106457606 +0000 UTC"
Jul 15 13:29:17.041550 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:17.041538    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12885h44m43.064922755s"
Jul 15 13:29:17.122888 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:17.122796    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:17.223507 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:17.223471    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:17.322937 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:17.322905    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01437e717367c58578eaaf88d68b2a5.slice/crio-90c3fa79a376f4aaebea83092cc7cf420b350f40cbb5012e0183428e87eb8ecb WatchSource:0}: Error finding container 90c3fa79a376f4aaebea83092cc7cf420b350f40cbb5012e0183428e87eb8ecb: Status 404 returned error can't find the container with id 90c3fa79a376f4aaebea83092cc7cf420b350f40cbb5012e0183428e87eb8ecb
Jul 15 13:29:17.323549 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:17.323532    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:17.323855 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:17.323836    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2238e507d170fbad58cb43abb1c04a.slice/crio-d4720e8e8e003e8100149ced70f5607177878d1e53d034284ed1215f6f6f1577 WatchSource:0}: Error finding container d4720e8e8e003e8100149ced70f5607177878d1e53d034284ed1215f6f6f1577: Status 404 returned error can't find the container with id d4720e8e8e003e8100149ced70f5607177878d1e53d034284ed1215f6f6f1577
Jul 15 13:29:17.328446 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:17.328431    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 13:29:17.426138 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:17.426034    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-134.ec2.internal\" not found"
Jul 15 13:29:17.443032 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:17.443006    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:29:17.451464 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:17.451439    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:29:17.529035 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:17.529013    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:17.541980 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:17.541960    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 13:29:17.543078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:17.543063    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal"
Jul 15 13:29:17.554789 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:17.554763    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 13:29:18.011535 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.011500    2577 apiserver.go:52] "Watching apiserver"
Jul 15 13:29:18.021321 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.019861    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 13:29:18.021321 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.021088    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-bk9wk","openshift-network-operator/iptables-alerter-djvv5","kube-system/global-pull-secret-syncer-xzbd5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q","openshift-cluster-node-tuning-operator/tuned-swrhf","openshift-multus/network-metrics-daemon-b8ddh","openshift-network-diagnostics/network-check-target-548bh","openshift-ovn-kubernetes/ovnkube-node-jvcml","kube-system/konnectivity-agent-6mkwn","kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal","openshift-dns/node-resolver-gdtmh","openshift-image-registry/node-ca-w7dvt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal","openshift-multus/multus-additional-cni-plugins-d6kv7"]
Jul 15 13:29:18.023531 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.023509    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:18.023664 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.023614    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:18.024572 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.024545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.025687 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.025656    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:18.025793 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.025745    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:18.027457 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.027436    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.027570 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.027498    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 13:29:18.028140 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.028116    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qk7h9\""
Jul 15 13:29:18.028247 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.028125    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 13:29:18.030636 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.028642    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.030636 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.029210    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 13:29:18.030636 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.030311    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 13:29:18.030636 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.030408    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qwzhg\""
Jul 15 13:29:18.030636 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.030510    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 13:29:18.030636 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.030582    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:18.030636 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.030636    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:18.031017 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.030782    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 13:29:18.031017 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.030973    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 13:29:18.031393 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.031370    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 13:29:18.031393 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.031384    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-htw5m\""
Jul 15 13:29:18.033259 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.033237    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:18.035326 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.035229    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.036396 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.036374    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.036942 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.036899    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 13:29:18.036942 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.036927    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7pngg\""
Jul 15 13:29:18.037561 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.037542    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 13:29:18.037931 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.037912    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 13:29:18.038020 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.037922    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 13:29:18.038080 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.038066    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 13:29:18.038307 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.038286    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 13:29:18.038425 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.038365    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.038425 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.038391    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 13:29:18.038880 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.038558    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8lnwl\""
Jul 15 13:29:18.039652 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.039625    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 13:29:18.039760 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.039668    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 13:29:18.039760 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.039706    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 13:29:18.039878 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.039779    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-blkbb\""
Jul 15 13:29:18.039998 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.039977    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.040870 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.040849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-sysctl-conf\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.040972 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.040906    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-host\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.040972 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.040932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e918381b-1b9b-414a-80cb-a2aeac624753-dbus\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:18.040972 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.040957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-socket-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.041153 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-device-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.041153 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.041153 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-sysctl-d\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.041311 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-sys\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.041311 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041213    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e918381b-1b9b-414a-80cb-a2aeac624753-kubelet-config\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:18.041311 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-sys-fs\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.041311 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-modprobe-d\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.041311 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-var-lib-kubelet\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.041594 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041416    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.041594 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041492    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cc6ada4d-9470-49d4-a669-44113e294c03-etc-tuned\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.041594 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041520    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc6ada4d-9470-49d4-a669-44113e294c03-tmp\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.041594 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-registration-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.041594 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041579    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.041835 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-kubernetes\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.041835 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041739    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 13:29:18.041835 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:18.041835 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-lib-modules\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.041835 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lggv4\" (UniqueName: \"kubernetes.io/projected/cc6ada4d-9470-49d4-a669-44113e294c03-kube-api-access-lggv4\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.042078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpbqc\" (UniqueName: \"kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc\") pod \"network-check-target-548bh\" (UID: \"1bfa268c-edc9-4417-a838-d777e9b9e940\") " pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:18.042078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041862    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 13:29:18.042078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042022    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 13:29:18.042226 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.041864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9hm2\" (UniqueName: \"kubernetes.io/projected/84c6a2f9-a280-4f19-bb68-4699778508ca-kube-api-access-j9hm2\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.042226 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:18.042226 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-systemd\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.042423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfch8\" (UniqueName: \"kubernetes.io/projected/7ce063d2-56c7-4d5b-b506-efee6758d32c-kube-api-access-tfch8\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:18.042423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cedb62b6-fd1f-4ccc-9b9e-987f001a3879-hosts-file\") pod \"node-resolver-gdtmh\" (UID: \"cedb62b6-fd1f-4ccc-9b9e-987f001a3879\") " pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.042423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042265    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 13:29:18.042423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042269    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 13:29:18.042423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrpbq\" (UniqueName: \"kubernetes.io/projected/cedb62b6-fd1f-4ccc-9b9e-987f001a3879-kube-api-access-jrpbq\") pod \"node-resolver-gdtmh\" (UID: \"cedb62b6-fd1f-4ccc-9b9e-987f001a3879\") " pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.042423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-sysconfig\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.042423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-run\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.042423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cedb62b6-fd1f-4ccc-9b9e-987f001a3879-tmp-dir\") pod \"node-resolver-gdtmh\" (UID: \"cedb62b6-fd1f-4ccc-9b9e-987f001a3879\") " pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.042805 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042463    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 13:29:18.042805 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042553    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nchvk\""
Jul 15 13:29:18.042889 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.042849    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 13:29:18.044953 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.043457    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 13:29:18.044953 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.043667    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lczg2\""
Jul 15 13:29:18.044953 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.044032    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 13:29:18.045616 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.045596    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 13:29:18.045851 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.045828    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z82q7\""
Jul 15 13:29:18.046133 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.046114    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 13:29:18.069369 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.069209    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s5t2n"
Jul 15 13:29:18.077670 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.077644    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s5t2n"
Jul 15 13:29:18.130788 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.130758    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 13:29:18.142641 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142584    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-898pv\" (UniqueName: \"kubernetes.io/projected/5c1ad8c2-e440-496a-b387-7fccfdfd5221-kube-api-access-898pv\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.142811 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142657    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-var-lib-cni-bin\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.142811 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-etc-kubernetes\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.142811 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-sys\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.142811 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e918381b-1b9b-414a-80cb-a2aeac624753-kubelet-config\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:18.142811 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-sys-fs\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8062b1d7-6402-4374-ae45-de07b61247a3-ovnkube-script-lib\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-sys\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx8vb\" (UniqueName: \"kubernetes.io/projected/8062b1d7-6402-4374-ae45-de07b61247a3-kube-api-access-fx8vb\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-os-release\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-hostroot\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e918381b-1b9b-414a-80cb-a2aeac624753-kubelet-config\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-var-lib-kubelet\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc6ada4d-9470-49d4-a669-44113e294c03-tmp\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-run-openvswitch\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-system-cni-dir\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-cnibin\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.142913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-sys-fs\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-multus-conf-dir\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.143078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143005    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-var-lib-kubelet\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143146    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-kubelet\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-run-netns\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8ed5e0f-ce91-4abf-afc9-e5425a797d09-iptables-alerter-script\") pod \"iptables-alerter-djvv5\" (UID: \"a8ed5e0f-ce91-4abf-afc9-e5425a797d09\") " pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-var-lib-kubelet\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-lib-modules\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143352    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbqc\" (UniqueName: \"kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc\") pod \"network-check-target-548bh\" (UID: \"1bfa268c-edc9-4417-a838-d777e9b9e940\") " pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143380    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-systemd-units\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8062b1d7-6402-4374-ae45-de07b61247a3-ovnkube-config\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143421    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143442    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8062b1d7-6402-4374-ae45-de07b61247a3-ovn-node-metrics-cert\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-systemd\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfch8\" (UniqueName: \"kubernetes.io/projected/7ce063d2-56c7-4d5b-b506-efee6758d32c-kube-api-access-tfch8\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cedb62b6-fd1f-4ccc-9b9e-987f001a3879-hosts-file\") pod \"node-resolver-gdtmh\" (UID: \"cedb62b6-fd1f-4ccc-9b9e-987f001a3879\") " pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrpbq\" (UniqueName: \"kubernetes.io/projected/cedb62b6-fd1f-4ccc-9b9e-987f001a3879-kube-api-access-jrpbq\") pod \"node-resolver-gdtmh\" (UID: \"cedb62b6-fd1f-4ccc-9b9e-987f001a3879\") " pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-log-socket\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.143792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143654    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-lib-modules\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c79e9a2e-64e5-487c-b25b-07b6853db2cf-host\") pod \"node-ca-w7dvt\" (UID: \"c79e9a2e-64e5-487c-b25b-07b6853db2cf\") " pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c79e9a2e-64e5-487c-b25b-07b6853db2cf-serviceca\") pod \"node-ca-w7dvt\" (UID: \"c79e9a2e-64e5-487c-b25b-07b6853db2cf\") " pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-systemd\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24422\" (UniqueName: \"kubernetes.io/projected/a8ed5e0f-ce91-4abf-afc9-e5425a797d09-kube-api-access-24422\") pod \"iptables-alerter-djvv5\" (UID: \"a8ed5e0f-ce91-4abf-afc9-e5425a797d09\") " pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-run-k8s-cni-cncf-io\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-run-systemd\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8ed5e0f-ce91-4abf-afc9-e5425a797d09-host-slash\") pod \"iptables-alerter-djvv5\" (UID: \"a8ed5e0f-ce91-4abf-afc9-e5425a797d09\") " pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-cnibin\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-run-netns\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.143986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-sysctl-conf\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-slash\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.144047    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.144118    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs podName:7ce063d2-56c7-4d5b-b506-efee6758d32c nodeName:}" failed. No retries permitted until 2026-07-15 13:29:18.644096768 +0000 UTC m=+2.987031830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs") pod "network-metrics-daemon-b8ddh" (UID: "7ce063d2-56c7-4d5b-b506-efee6758d32c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cedb62b6-fd1f-4ccc-9b9e-987f001a3879-hosts-file\") pod \"node-resolver-gdtmh\" (UID: \"cedb62b6-fd1f-4ccc-9b9e-987f001a3879\") " pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-sysctl-conf\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144045    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-etc-openvswitch\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.144716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144321    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rv8l\" (UniqueName: \"kubernetes.io/projected/c79e9a2e-64e5-487c-b25b-07b6853db2cf-kube-api-access-8rv8l\") pod \"node-ca-w7dvt\" (UID: \"c79e9a2e-64e5-487c-b25b-07b6853db2cf\") " pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c1ad8c2-e440-496a-b387-7fccfdfd5221-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144422    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-system-cni-dir\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-os-release\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144533    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-sysctl-d\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-multus-cni-dir\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144577    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-multus-socket-dir-parent\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144665    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-sysctl-d\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-modprobe-d\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cc6ada4d-9470-49d4-a669-44113e294c03-etc-tuned\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-registration-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-modprobe-d\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.145612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-kubernetes\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144920    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-var-lib-openvswitch\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-cni-bin\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.144978    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/160c3910-c9da-44fd-a1c9-22fb19468b3d-konnectivity-ca\") pod \"konnectivity-agent-6mkwn\" (UID: \"160c3910-c9da-44fd-a1c9-22fb19468b3d\") " pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-run-multus-certs\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lggv4\" (UniqueName: \"kubernetes.io/projected/cc6ada4d-9470-49d4-a669-44113e294c03-kube-api-access-lggv4\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-kubernetes\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9hm2\" (UniqueName: \"kubernetes.io/projected/84c6a2f9-a280-4f19-bb68-4699778508ca-kube-api-access-j9hm2\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-run-ovn\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145129    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/160c3910-c9da-44fd-a1c9-22fb19468b3d-agent-certs\") pod \"konnectivity-agent-6mkwn\" (UID: \"160c3910-c9da-44fd-a1c9-22fb19468b3d\") " pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c1ad8c2-e440-496a-b387-7fccfdfd5221-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145242    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-cni-netd\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3f867666-5b1a-4502-b9b7-b043013554d6-multus-daemon-config\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-sysconfig\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.146495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-run\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-node-log\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.145465    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145512    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8062b1d7-6402-4374-ae45-de07b61247a3-env-overrides\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f867666-5b1a-4502-b9b7-b043013554d6-cni-binary-copy\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-var-lib-cni-multus\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-run\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf9kl\" (UniqueName: \"kubernetes.io/projected/3f867666-5b1a-4502-b9b7-b043013554d6-kube-api-access-wf9kl\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cedb62b6-fd1f-4ccc-9b9e-987f001a3879-tmp-dir\") pod \"node-resolver-gdtmh\" (UID: \"cedb62b6-fd1f-4ccc-9b9e-987f001a3879\") " pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-etc-sysconfig\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145661    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-host\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e918381b-1b9b-414a-80cb-a2aeac624753-dbus\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-socket-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-device-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145826    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c1ad8c2-e440-496a-b387-7fccfdfd5221-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145923    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc6ada4d-9470-49d4-a669-44113e294c03-host\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145928    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-registration-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.145991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-device-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.147253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.146100    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e918381b-1b9b-414a-80cb-a2aeac624753-dbus\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:18.148026 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.146229    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret podName:e918381b-1b9b-414a-80cb-a2aeac624753 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:18.646207992 +0000 UTC m=+2.989143055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret") pod "global-pull-secret-syncer-xzbd5" (UID: "e918381b-1b9b-414a-80cb-a2aeac624753") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:18.148026 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.146306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84c6a2f9-a280-4f19-bb68-4699778508ca-socket-dir\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.148026 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.146536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cedb62b6-fd1f-4ccc-9b9e-987f001a3879-tmp-dir\") pod \"node-resolver-gdtmh\" (UID: \"cedb62b6-fd1f-4ccc-9b9e-987f001a3879\") " pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.148375 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.148350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cc6ada4d-9470-49d4-a669-44113e294c03-etc-tuned\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.148621 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.148572    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal" event={"ID":"a01437e717367c58578eaaf88d68b2a5","Type":"ContainerStarted","Data":"90c3fa79a376f4aaebea83092cc7cf420b350f40cbb5012e0183428e87eb8ecb"}
Jul 15 13:29:18.150266 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.150022    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc6ada4d-9470-49d4-a669-44113e294c03-tmp\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.151111 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.151083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal" event={"ID":"ae2238e507d170fbad58cb43abb1c04a","Type":"ContainerStarted","Data":"d4720e8e8e003e8100149ced70f5607177878d1e53d034284ed1215f6f6f1577"}
Jul 15 13:29:18.155627 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.155529    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:29:18.155627 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.155554    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:29:18.155627 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.155569    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lpbqc for pod openshift-network-diagnostics/network-check-target-548bh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:18.155876 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.155648    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc podName:1bfa268c-edc9-4417-a838-d777e9b9e940 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:18.655627711 +0000 UTC m=+2.998562788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lpbqc" (UniqueName: "kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc") pod "network-check-target-548bh" (UID: "1bfa268c-edc9-4417-a838-d777e9b9e940") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:18.158564 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.158440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfch8\" (UniqueName: \"kubernetes.io/projected/7ce063d2-56c7-4d5b-b506-efee6758d32c-kube-api-access-tfch8\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:18.158564 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.158513    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lggv4\" (UniqueName: \"kubernetes.io/projected/cc6ada4d-9470-49d4-a669-44113e294c03-kube-api-access-lggv4\") pod \"tuned-swrhf\" (UID: \"cc6ada4d-9470-49d4-a669-44113e294c03\") " pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.158721 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.158662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9hm2\" (UniqueName: \"kubernetes.io/projected/84c6a2f9-a280-4f19-bb68-4699778508ca-kube-api-access-j9hm2\") pod \"aws-ebs-csi-driver-node-67v6q\" (UID: \"84c6a2f9-a280-4f19-bb68-4699778508ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.159761 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.159732    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrpbq\" (UniqueName: \"kubernetes.io/projected/cedb62b6-fd1f-4ccc-9b9e-987f001a3879-kube-api-access-jrpbq\") pod \"node-resolver-gdtmh\" (UID: \"cedb62b6-fd1f-4ccc-9b9e-987f001a3879\") " pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.246651 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c1ad8c2-e440-496a-b387-7fccfdfd5221-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.246651 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246657    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-system-cni-dir\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.246905 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-os-release\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.246905 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-multus-cni-dir\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.246905 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-multus-socket-dir-parent\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.246905 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.246905 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-var-lib-openvswitch\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.246905 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-cni-bin\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.246905 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/160c3910-c9da-44fd-a1c9-22fb19468b3d-konnectivity-ca\") pod \"konnectivity-agent-6mkwn\" (UID: \"160c3910-c9da-44fd-a1c9-22fb19468b3d\") " pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:18.246905 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-run-multus-certs\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-run-ovn\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/160c3910-c9da-44fd-a1c9-22fb19468b3d-agent-certs\") pod \"konnectivity-agent-6mkwn\" (UID: \"160c3910-c9da-44fd-a1c9-22fb19468b3d\") " pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.246999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c1ad8c2-e440-496a-b387-7fccfdfd5221-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-cni-netd\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3f867666-5b1a-4502-b9b7-b043013554d6-multus-daemon-config\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-node-log\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8062b1d7-6402-4374-ae45-de07b61247a3-env-overrides\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f867666-5b1a-4502-b9b7-b043013554d6-cni-binary-copy\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-var-lib-cni-multus\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247206    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c1ad8c2-e440-496a-b387-7fccfdfd5221-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf9kl\" (UniqueName: \"kubernetes.io/projected/3f867666-5b1a-4502-b9b7-b043013554d6-kube-api-access-wf9kl\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247256    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c1ad8c2-e440-496a-b387-7fccfdfd5221-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-898pv\" (UniqueName: \"kubernetes.io/projected/5c1ad8c2-e440-496a-b387-7fccfdfd5221-kube-api-access-898pv\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-var-lib-cni-bin\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-run-ovn\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.247361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247359    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-etc-kubernetes\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.248067 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-system-cni-dir\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.248067 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247425    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-etc-kubernetes\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.248067 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247428    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-var-lib-openvswitch\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.248067 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-var-lib-cni-multus\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.248067 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247468    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-cni-bin\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.248067 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.248067 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8062b1d7-6402-4374-ae45-de07b61247a3-env-overrides\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.248067 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-os-release\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.248067 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247915    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-var-lib-cni-bin\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.251491 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.251462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f867666-5b1a-4502-b9b7-b043013554d6-cni-binary-copy\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.251963 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.251935    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c1ad8c2-e440-496a-b387-7fccfdfd5221-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.252038 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.252014    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-run-multus-certs\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.252477 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.252406    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c1ad8c2-e440-496a-b387-7fccfdfd5221-cni-binary-copy\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.252477 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.252424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/160c3910-c9da-44fd-a1c9-22fb19468b3d-konnectivity-ca\") pod \"konnectivity-agent-6mkwn\" (UID: \"160c3910-c9da-44fd-a1c9-22fb19468b3d\") " pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:18.252661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.252480    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-multus-cni-dir\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.252661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.247928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8062b1d7-6402-4374-ae45-de07b61247a3-ovnkube-script-lib\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.252661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.252596    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-node-log\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.253123 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3f867666-5b1a-4502-b9b7-b043013554d6-multus-daemon-config\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.253402 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8062b1d7-6402-4374-ae45-de07b61247a3-ovnkube-script-lib\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.253492 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-multus-socket-dir-parent\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.253564 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-cni-netd\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.253615 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-run-ovn-kubernetes\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.253667 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx8vb\" (UniqueName: \"kubernetes.io/projected/8062b1d7-6402-4374-ae45-de07b61247a3-kube-api-access-fx8vb\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.253714 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-os-release\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.253714 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-hostroot\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.253803 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253736    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-run-openvswitch\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.253803 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-system-cni-dir\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.253904 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-cnibin\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.253904 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253830    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-multus-conf-dir\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.253904 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-kubelet\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.254045 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-run-netns\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.254045 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8ed5e0f-ce91-4abf-afc9-e5425a797d09-iptables-alerter-script\") pod \"iptables-alerter-djvv5\" (UID: \"a8ed5e0f-ce91-4abf-afc9-e5425a797d09\") " pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.254045 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.253980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-var-lib-kubelet\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.254045 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-systemd-units\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.254238 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8062b1d7-6402-4374-ae45-de07b61247a3-ovnkube-config\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.254238 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8062b1d7-6402-4374-ae45-de07b61247a3-ovn-node-metrics-cert\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.254238 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-log-socket\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.254238 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c79e9a2e-64e5-487c-b25b-07b6853db2cf-host\") pod \"node-ca-w7dvt\" (UID: \"c79e9a2e-64e5-487c-b25b-07b6853db2cf\") " pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.254238 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c79e9a2e-64e5-487c-b25b-07b6853db2cf-serviceca\") pod \"node-ca-w7dvt\" (UID: \"c79e9a2e-64e5-487c-b25b-07b6853db2cf\") " pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.254238 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24422\" (UniqueName: \"kubernetes.io/projected/a8ed5e0f-ce91-4abf-afc9-e5425a797d09-kube-api-access-24422\") pod \"iptables-alerter-djvv5\" (UID: \"a8ed5e0f-ce91-4abf-afc9-e5425a797d09\") " pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.254530 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-run-k8s-cni-cncf-io\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.254530 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254299    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-run-systemd\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.254530 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254329    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8ed5e0f-ce91-4abf-afc9-e5425a797d09-host-slash\") pod \"iptables-alerter-djvv5\" (UID: \"a8ed5e0f-ce91-4abf-afc9-e5425a797d09\") " pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.254530 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254422    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8ed5e0f-ce91-4abf-afc9-e5425a797d09-host-slash\") pod \"iptables-alerter-djvv5\" (UID: \"a8ed5e0f-ce91-4abf-afc9-e5425a797d09\") " pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.254530 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-var-lib-kubelet\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.254789 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254532    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-systemd-units\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254922    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-run-k8s-cni-cncf-io\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8ed5e0f-ce91-4abf-afc9-e5425a797d09-iptables-alerter-script\") pod \"iptables-alerter-djvv5\" (UID: \"a8ed5e0f-ce91-4abf-afc9-e5425a797d09\") " pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.254947    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-system-cni-dir\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-cnibin\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255080    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-multus-conf-dir\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-kubelet\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255169    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8062b1d7-6402-4374-ae45-de07b61247a3-ovnkube-config\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c1ad8c2-e440-496a-b387-7fccfdfd5221-os-release\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-run-netns\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255251    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-hostroot\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255275    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-run-openvswitch\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c79e9a2e-64e5-487c-b25b-07b6853db2cf-host\") pod \"node-ca-w7dvt\" (UID: \"c79e9a2e-64e5-487c-b25b-07b6853db2cf\") " pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255318    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-log-socket\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255367    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c79e9a2e-64e5-487c-b25b-07b6853db2cf-serviceca\") pod \"node-ca-w7dvt\" (UID: \"c79e9a2e-64e5-487c-b25b-07b6853db2cf\") " pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255373    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-run-systemd\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/160c3910-c9da-44fd-a1c9-22fb19468b3d-agent-certs\") pod \"konnectivity-agent-6mkwn\" (UID: \"160c3910-c9da-44fd-a1c9-22fb19468b3d\") " pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:18.255413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-cnibin\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.256441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-cnibin\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.256441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-run-netns\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.256441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-slash\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.256441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-etc-openvswitch\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.256441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255584    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f867666-5b1a-4502-b9b7-b043013554d6-host-run-netns\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.256441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.256441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255605    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-slash\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.256441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rv8l\" (UniqueName: \"kubernetes.io/projected/c79e9a2e-64e5-487c-b25b-07b6853db2cf-kube-api-access-8rv8l\") pod \"node-ca-w7dvt\" (UID: \"c79e9a2e-64e5-487c-b25b-07b6853db2cf\") " pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.256441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255670    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-etc-openvswitch\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.256441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.255680    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8062b1d7-6402-4374-ae45-de07b61247a3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.259566 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.259196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8062b1d7-6402-4374-ae45-de07b61247a3-ovn-node-metrics-cert\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.260044 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.260010    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf9kl\" (UniqueName: \"kubernetes.io/projected/3f867666-5b1a-4502-b9b7-b043013554d6-kube-api-access-wf9kl\") pod \"multus-bk9wk\" (UID: \"3f867666-5b1a-4502-b9b7-b043013554d6\") " pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.263568 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.263506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-898pv\" (UniqueName: \"kubernetes.io/projected/5c1ad8c2-e440-496a-b387-7fccfdfd5221-kube-api-access-898pv\") pod \"multus-additional-cni-plugins-d6kv7\" (UID: \"5c1ad8c2-e440-496a-b387-7fccfdfd5221\") " pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.266187 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.264428    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx8vb\" (UniqueName: \"kubernetes.io/projected/8062b1d7-6402-4374-ae45-de07b61247a3-kube-api-access-fx8vb\") pod \"ovnkube-node-jvcml\" (UID: \"8062b1d7-6402-4374-ae45-de07b61247a3\") " pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.266187 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.265400    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24422\" (UniqueName: \"kubernetes.io/projected/a8ed5e0f-ce91-4abf-afc9-e5425a797d09-kube-api-access-24422\") pod \"iptables-alerter-djvv5\" (UID: \"a8ed5e0f-ce91-4abf-afc9-e5425a797d09\") " pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.266483 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.266464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rv8l\" (UniqueName: \"kubernetes.io/projected/c79e9a2e-64e5-487c-b25b-07b6853db2cf-kube-api-access-8rv8l\") pod \"node-ca-w7dvt\" (UID: \"c79e9a2e-64e5-487c-b25b-07b6853db2cf\") " pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.341137 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.341091    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gdtmh"
Jul 15 13:29:18.349529 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.349491    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q"
Jul 15 13:29:18.357159 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:18.357130    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcedb62b6_fd1f_4ccc_9b9e_987f001a3879.slice/crio-0688c10e65fc9b23cb7f48280d3334c731812311f993b3142806a65c439e78dc WatchSource:0}: Error finding container 0688c10e65fc9b23cb7f48280d3334c731812311f993b3142806a65c439e78dc: Status 404 returned error can't find the container with id 0688c10e65fc9b23cb7f48280d3334c731812311f993b3142806a65c439e78dc
Jul 15 13:29:18.362141 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.361901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-swrhf"
Jul 15 13:29:18.373629 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:18.373596    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6ada4d_9470_49d4_a669_44113e294c03.slice/crio-03f985c2fc76b494ebeb4254ffefb0e8f285d29778297edfc65e1cc07de36218 WatchSource:0}: Error finding container 03f985c2fc76b494ebeb4254ffefb0e8f285d29778297edfc65e1cc07de36218: Status 404 returned error can't find the container with id 03f985c2fc76b494ebeb4254ffefb0e8f285d29778297edfc65e1cc07de36218
Jul 15 13:29:18.375111 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.374926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:18.380568 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.380524    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d6kv7"
Jul 15 13:29:18.390984 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.390814    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-djvv5"
Jul 15 13:29:18.394207 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:18.394172    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1ad8c2_e440_496a_b387_7fccfdfd5221.slice/crio-6b37da73e6ce859fe5eaad9bb9cb5f8cc8d40f34a01715ab0af750d365182427 WatchSource:0}: Error finding container 6b37da73e6ce859fe5eaad9bb9cb5f8cc8d40f34a01715ab0af750d365182427: Status 404 returned error can't find the container with id 6b37da73e6ce859fe5eaad9bb9cb5f8cc8d40f34a01715ab0af750d365182427
Jul 15 13:29:18.398253 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.398105    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:18.401595 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:18.401567    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ed5e0f_ce91_4abf_afc9_e5425a797d09.slice/crio-bc50401cf38207892ab48c4bfb7b258e03824be3765fdd684e0c50e1415e939d WatchSource:0}: Error finding container bc50401cf38207892ab48c4bfb7b258e03824be3765fdd684e0c50e1415e939d: Status 404 returned error can't find the container with id bc50401cf38207892ab48c4bfb7b258e03824be3765fdd684e0c50e1415e939d
Jul 15 13:29:18.406127 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.405859    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bk9wk"
Jul 15 13:29:18.408791 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:18.408760    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8062b1d7_6402_4374_ae45_de07b61247a3.slice/crio-1450fa827a05704f669101c450a26cf1792db44c49adaaebabfeadeef08129ea WatchSource:0}: Error finding container 1450fa827a05704f669101c450a26cf1792db44c49adaaebabfeadeef08129ea: Status 404 returned error can't find the container with id 1450fa827a05704f669101c450a26cf1792db44c49adaaebabfeadeef08129ea
Jul 15 13:29:18.413824 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.413802    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w7dvt"
Jul 15 13:29:18.425401 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:29:18.425362    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc79e9a2e_64e5_487c_b25b_07b6853db2cf.slice/crio-ac43356604276d04462a349b933da292e93d0d506421ecd9f23827d6bd06f2f5 WatchSource:0}: Error finding container ac43356604276d04462a349b933da292e93d0d506421ecd9f23827d6bd06f2f5: Status 404 returned error can't find the container with id ac43356604276d04462a349b933da292e93d0d506421ecd9f23827d6bd06f2f5
Jul 15 13:29:18.543511 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.543422    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:29:18.658420 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.658377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:18.658621 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.658429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbqc\" (UniqueName: \"kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc\") pod \"network-check-target-548bh\" (UID: \"1bfa268c-edc9-4417-a838-d777e9b9e940\") " pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:18.658621 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:18.658483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:18.658621 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.658549    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:18.658621 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.658584    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:18.658621 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.658608    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:29:18.658621 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.658622    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs podName:7ce063d2-56c7-4d5b-b506-efee6758d32c nodeName:}" failed. No retries permitted until 2026-07-15 13:29:19.658601993 +0000 UTC m=+4.001537066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs") pod "network-metrics-daemon-b8ddh" (UID: "7ce063d2-56c7-4d5b-b506-efee6758d32c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:18.658948 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.658626    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:29:18.658948 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.658640    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lpbqc for pod openshift-network-diagnostics/network-check-target-548bh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:18.658948 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.658645    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret podName:e918381b-1b9b-414a-80cb-a2aeac624753 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:19.658634774 +0000 UTC m=+4.001569837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret") pod "global-pull-secret-syncer-xzbd5" (UID: "e918381b-1b9b-414a-80cb-a2aeac624753") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:18.658948 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:18.658677    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc podName:1bfa268c-edc9-4417-a838-d777e9b9e940 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:19.658664523 +0000 UTC m=+4.001599585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpbqc" (UniqueName: "kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc") pod "network-check-target-548bh" (UID: "1bfa268c-edc9-4417-a838-d777e9b9e940") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:19.080695 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.080633    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 13:24:18 +0000 UTC" deadline="2027-12-20 09:05:57.862984592 +0000 UTC"
Jul 15 13:29:19.080695 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.080692    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12547h36m38.782298404s"
Jul 15 13:29:19.153985 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.153899    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q" event={"ID":"84c6a2f9-a280-4f19-bb68-4699778508ca","Type":"ContainerStarted","Data":"f8cdb093fe1f2d78065b115ed80243e01db4ff91786b99209801bac6c1a78a41"}
Jul 15 13:29:19.155129 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.155092    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gdtmh" event={"ID":"cedb62b6-fd1f-4ccc-9b9e-987f001a3879","Type":"ContainerStarted","Data":"0688c10e65fc9b23cb7f48280d3334c731812311f993b3142806a65c439e78dc"}
Jul 15 13:29:19.156208 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.156177    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bk9wk" event={"ID":"3f867666-5b1a-4502-b9b7-b043013554d6","Type":"ContainerStarted","Data":"0f7189804f3bad9658b71f6918ee0ef7f0e1f30b4a707a7a515bd59843a07ad0"}
Jul 15 13:29:19.157601 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.157560    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerStarted","Data":"1450fa827a05704f669101c450a26cf1792db44c49adaaebabfeadeef08129ea"}
Jul 15 13:29:19.158959 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.158919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-djvv5" event={"ID":"a8ed5e0f-ce91-4abf-afc9-e5425a797d09","Type":"ContainerStarted","Data":"bc50401cf38207892ab48c4bfb7b258e03824be3765fdd684e0c50e1415e939d"}
Jul 15 13:29:19.160150 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.160121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-swrhf" event={"ID":"cc6ada4d-9470-49d4-a669-44113e294c03","Type":"ContainerStarted","Data":"03f985c2fc76b494ebeb4254ffefb0e8f285d29778297edfc65e1cc07de36218"}
Jul 15 13:29:19.161432 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.161407    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w7dvt" event={"ID":"c79e9a2e-64e5-487c-b25b-07b6853db2cf","Type":"ContainerStarted","Data":"ac43356604276d04462a349b933da292e93d0d506421ecd9f23827d6bd06f2f5"}
Jul 15 13:29:19.162619 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.162592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6kv7" event={"ID":"5c1ad8c2-e440-496a-b387-7fccfdfd5221","Type":"ContainerStarted","Data":"6b37da73e6ce859fe5eaad9bb9cb5f8cc8d40f34a01715ab0af750d365182427"}
Jul 15 13:29:19.163640 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.163619    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6mkwn" event={"ID":"160c3910-c9da-44fd-a1c9-22fb19468b3d","Type":"ContainerStarted","Data":"d20dfbae00bbd3c10a6135e568f2671b5394b3121d485eb274668306d64b9b56"}
Jul 15 13:29:19.665830 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.665768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:19.666046 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.665853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:19.666046 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:19.665903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbqc\" (UniqueName: \"kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc\") pod \"network-check-target-548bh\" (UID: \"1bfa268c-edc9-4417-a838-d777e9b9e940\") " pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:19.666154 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:19.666062    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:29:19.666154 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:19.666081    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:29:19.666154 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:19.666094    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lpbqc for pod openshift-network-diagnostics/network-check-target-548bh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:19.666355 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:19.666159    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc podName:1bfa268c-edc9-4417-a838-d777e9b9e940 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:21.666140552 +0000 UTC m=+6.009075613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpbqc" (UniqueName: "kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc") pod "network-check-target-548bh" (UID: "1bfa268c-edc9-4417-a838-d777e9b9e940") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:19.666635 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:19.666613    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:19.666704 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:19.666692    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret podName:e918381b-1b9b-414a-80cb-a2aeac624753 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:21.666661484 +0000 UTC m=+6.009596545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret") pod "global-pull-secret-syncer-xzbd5" (UID: "e918381b-1b9b-414a-80cb-a2aeac624753") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:19.666767 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:19.666758    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:19.666827 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:19.666795    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs podName:7ce063d2-56c7-4d5b-b506-efee6758d32c nodeName:}" failed. No retries permitted until 2026-07-15 13:29:21.666779942 +0000 UTC m=+6.009715000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs") pod "network-metrics-daemon-b8ddh" (UID: "7ce063d2-56c7-4d5b-b506-efee6758d32c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:20.081802 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:20.081694    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 13:24:18 +0000 UTC" deadline="2028-04-11 21:34:37.694376262 +0000 UTC"
Jul 15 13:29:20.081802 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:20.081731    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15272h5m17.612649559s"
Jul 15 13:29:20.092634 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:20.092320    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 13:29:20.144420 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:20.143587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:20.144420 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:20.143714    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:20.144420 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:20.144104    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:20.144420 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:20.144209    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:20.144420 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:20.144293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:20.144420 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:20.144368    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:21.197942 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:21.197667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal" event={"ID":"a01437e717367c58578eaaf88d68b2a5","Type":"ContainerStarted","Data":"7c91dd39d8d8a85ea7efb73c518c9f8dc0f736ce6aec6fc2dba22b6bb0f65646"}
Jul 15 13:29:21.206888 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:21.206846    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae2238e507d170fbad58cb43abb1c04a" containerID="eef752aef3ee092d5b5c7cc2501f05b23df05af248f17c56bbe9e05bbcb78e16" exitCode=0
Jul 15 13:29:21.207071 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:21.206901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal" event={"ID":"ae2238e507d170fbad58cb43abb1c04a","Type":"ContainerDied","Data":"eef752aef3ee092d5b5c7cc2501f05b23df05af248f17c56bbe9e05bbcb78e16"}
Jul 15 13:29:21.216376 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:21.216317    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-134.ec2.internal" podStartSLOduration=4.216297097 podStartE2EDuration="4.216297097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:21.215959782 +0000 UTC m=+5.558894867" watchObservedRunningTime="2026-07-15 13:29:21.216297097 +0000 UTC m=+5.559232179"
Jul 15 13:29:21.681549 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:21.681514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:21.681730 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:21.681577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:21.681730 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:21.681606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbqc\" (UniqueName: \"kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc\") pod \"network-check-target-548bh\" (UID: \"1bfa268c-edc9-4417-a838-d777e9b9e940\") " pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:21.681856 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:21.681740    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:29:21.681856 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:21.681756    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:29:21.681856 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:21.681768    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lpbqc for pod openshift-network-diagnostics/network-check-target-548bh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:21.681856 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:21.681821    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc podName:1bfa268c-edc9-4417-a838-d777e9b9e940 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:25.681803783 +0000 UTC m=+10.024738844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpbqc" (UniqueName: "kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc") pod "network-check-target-548bh" (UID: "1bfa268c-edc9-4417-a838-d777e9b9e940") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:21.682227 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:21.682204    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:21.682310 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:21.682275    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret podName:e918381b-1b9b-414a-80cb-a2aeac624753 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:25.682243456 +0000 UTC m=+10.025178523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret") pod "global-pull-secret-syncer-xzbd5" (UID: "e918381b-1b9b-414a-80cb-a2aeac624753") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:21.682384 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:21.682338    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:21.682384 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:21.682371    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs podName:7ce063d2-56c7-4d5b-b506-efee6758d32c nodeName:}" failed. No retries permitted until 2026-07-15 13:29:25.682359014 +0000 UTC m=+10.025294078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs") pod "network-metrics-daemon-b8ddh" (UID: "7ce063d2-56c7-4d5b-b506-efee6758d32c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:22.142198 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:22.141386    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:22.142198 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:22.141538    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:22.142198 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:22.141955    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:22.142198 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:22.142064    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:22.142788 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:22.142622    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:22.142788 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:22.142720    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:24.143024 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:24.140809    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:24.143024 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:24.140950    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:24.143024 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:24.141341    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:24.143024 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:24.141440    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:24.143024 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:24.141484    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:24.143024 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:24.141556    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:25.719746 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:25.719703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:25.720183 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:25.719753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbqc\" (UniqueName: \"kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc\") pod \"network-check-target-548bh\" (UID: \"1bfa268c-edc9-4417-a838-d777e9b9e940\") " pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:25.720183 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:25.719806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:25.720183 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:25.719953    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:25.720183 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:25.720017    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret podName:e918381b-1b9b-414a-80cb-a2aeac624753 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:33.71999612 +0000 UTC m=+18.062931177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret") pod "global-pull-secret-syncer-xzbd5" (UID: "e918381b-1b9b-414a-80cb-a2aeac624753") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:25.720463 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:25.720443    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:25.720524 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:25.720501    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs podName:7ce063d2-56c7-4d5b-b506-efee6758d32c nodeName:}" failed. No retries permitted until 2026-07-15 13:29:33.720483785 +0000 UTC m=+18.063418845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs") pod "network-metrics-daemon-b8ddh" (UID: "7ce063d2-56c7-4d5b-b506-efee6758d32c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:25.720597 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:25.720583    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:29:25.720656 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:25.720605    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:29:25.720656 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:25.720617    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lpbqc for pod openshift-network-diagnostics/network-check-target-548bh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:25.720759 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:25.720657    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc podName:1bfa268c-edc9-4417-a838-d777e9b9e940 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:33.720644326 +0000 UTC m=+18.063579386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpbqc" (UniqueName: "kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc") pod "network-check-target-548bh" (UID: "1bfa268c-edc9-4417-a838-d777e9b9e940") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:26.145119 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:26.145020    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:26.145119 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:26.145049    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:26.145350 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:26.145130    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:26.145350 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:26.145178    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:26.145470 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:26.145249    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:26.145526 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:26.145419    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:28.141080 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:28.140632    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:28.141080 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:28.140631    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:28.141080 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:28.140871    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:28.141080 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:28.140752    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:28.141080 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:28.140926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:28.141080 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:28.141007    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:30.141792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:30.141748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:30.142183 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:30.141748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:30.142183 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:30.141873    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:30.142183 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:30.141953    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:30.142183 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:30.141748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:30.142183 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:30.142044    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:32.140927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:32.140893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:32.141429 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:32.140899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:32.141429 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:32.141029    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:32.141429 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:32.140899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:32.141429 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:32.141128    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:32.141429 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:32.141167    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:33.776563 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:33.776520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:33.777103 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:33.776609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:33.777103 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:33.776644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbqc\" (UniqueName: \"kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc\") pod \"network-check-target-548bh\" (UID: \"1bfa268c-edc9-4417-a838-d777e9b9e940\") " pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:33.777103 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:33.776663    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:33.777103 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:33.776740    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret podName:e918381b-1b9b-414a-80cb-a2aeac624753 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:49.776718573 +0000 UTC m=+34.119653637 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret") pod "global-pull-secret-syncer-xzbd5" (UID: "e918381b-1b9b-414a-80cb-a2aeac624753") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:33.777103 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:33.776763    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:33.777103 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:33.776782    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:29:33.777103 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:33.776804    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:29:33.777103 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:33.776818    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lpbqc for pod openshift-network-diagnostics/network-check-target-548bh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:33.777103 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:33.776818    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs podName:7ce063d2-56c7-4d5b-b506-efee6758d32c nodeName:}" failed. No retries permitted until 2026-07-15 13:29:49.776802922 +0000 UTC m=+34.119737985 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs") pod "network-metrics-daemon-b8ddh" (UID: "7ce063d2-56c7-4d5b-b506-efee6758d32c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:33.777103 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:33.776876    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc podName:1bfa268c-edc9-4417-a838-d777e9b9e940 nodeName:}" failed. No retries permitted until 2026-07-15 13:29:49.77686192 +0000 UTC m=+34.119796978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpbqc" (UniqueName: "kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc") pod "network-check-target-548bh" (UID: "1bfa268c-edc9-4417-a838-d777e9b9e940") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:34.141020 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:34.140938    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:34.141020 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:34.140960    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:34.141020 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:34.140946    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:34.141305 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:34.141074    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:34.141305 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:34.141120    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:34.141305 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:34.141190    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:36.141981 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:36.141939    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:36.142500 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:36.142029    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:36.142500 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:36.142129    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:36.142500 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:36.142162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:36.142500 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:36.142240    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:36.142500 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:36.142342    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:38.142286 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.141105    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:38.142286 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:38.141388    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:38.142286 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.141433    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:38.142286 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.141451    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:38.142286 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:38.141527    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:38.142286 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:38.141611    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:38.243691 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.243651    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal" event={"ID":"ae2238e507d170fbad58cb43abb1c04a","Type":"ContainerStarted","Data":"0716c6e58ff04a31ea60aeb8e61aeaa1ef32a2d3afafe4ad79fe448aa631141d"}
Jul 15 13:29:38.245393 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.245358    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bk9wk" event={"ID":"3f867666-5b1a-4502-b9b7-b043013554d6","Type":"ContainerStarted","Data":"8663ea46536fce20ac479253be7be32b1eed6c515f7e8be8bcaa64fd91fd26ff"}
Jul 15 13:29:38.247187 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.247160    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerStarted","Data":"f62a8e2129aa8c1944e40e999ae68e71165b20f91715af1e87db0852a06e7d86"}
Jul 15 13:29:38.247313 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.247225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerStarted","Data":"1e4609e822a94caaab9fdd2a17312a4f4f3245788bb63346746763b851282405"}
Jul 15 13:29:38.248568 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.248540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-swrhf" event={"ID":"cc6ada4d-9470-49d4-a669-44113e294c03","Type":"ContainerStarted","Data":"d2913dbfdbe1e1ee6f6378a828d19a731da0659448943dfc0ccf5da30a78c70f"}
Jul 15 13:29:38.250470 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.250447    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w7dvt" event={"ID":"c79e9a2e-64e5-487c-b25b-07b6853db2cf","Type":"ContainerStarted","Data":"2ef98967def28114c3c8697605d4fdd13ec4182c03ddaba1f037b04ad8f2856e"}
Jul 15 13:29:38.251679 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.251651    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c1ad8c2-e440-496a-b387-7fccfdfd5221" containerID="07f1b8803f920cd480a0c10696923b8eb415b97f8f414e9e65ad2990926e435b" exitCode=0
Jul 15 13:29:38.251748 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.251710    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6kv7" event={"ID":"5c1ad8c2-e440-496a-b387-7fccfdfd5221","Type":"ContainerDied","Data":"07f1b8803f920cd480a0c10696923b8eb415b97f8f414e9e65ad2990926e435b"}
Jul 15 13:29:38.253201 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.253171    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6mkwn" event={"ID":"160c3910-c9da-44fd-a1c9-22fb19468b3d","Type":"ContainerStarted","Data":"46493302e5707adab58f8ddaaa15b644521a5109f7416a32177c8c017d197fb3"}
Jul 15 13:29:38.255144 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.255120    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q" event={"ID":"84c6a2f9-a280-4f19-bb68-4699778508ca","Type":"ContainerStarted","Data":"174d4ab05e106b484afdb1e0ed4d3b3518117a3f12f571e5a4dac05125ea64b3"}
Jul 15 13:29:38.256423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.256402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gdtmh" event={"ID":"cedb62b6-fd1f-4ccc-9b9e-987f001a3879","Type":"ContainerStarted","Data":"fff16b4e585827f264479d2fe70a3e844a1e33cb48485161d4554bc44a870685"}
Jul 15 13:29:38.273431 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.273374    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-134.ec2.internal" podStartSLOduration=21.273354223 podStartE2EDuration="21.273354223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:38.273255486 +0000 UTC m=+22.616190567" watchObservedRunningTime="2026-07-15 13:29:38.273354223 +0000 UTC m=+22.616289304"
Jul 15 13:29:38.340992 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.340954    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-swrhf" podStartSLOduration=2.854179444 podStartE2EDuration="22.340938984s" totalImagesPullingTime="19.48675954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:38.340867072 +0000 UTC m=+22.683802152" watchObservedRunningTime="2026-07-15 13:29:38.340938984 +0000 UTC m=+22.683874063"
Jul 15 13:29:38.341321 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.341298    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gdtmh" podStartSLOduration=3.221610026 podStartE2EDuration="22.34128788s" totalImagesPullingTime="19.119677854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:38.305189125 +0000 UTC m=+22.648124220" watchObservedRunningTime="2026-07-15 13:29:38.34128788 +0000 UTC m=+22.684222951"
Jul 15 13:29:38.361219 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.361152    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6mkwn" podStartSLOduration=3.269659041 podStartE2EDuration="22.361136489s" totalImagesPullingTime="19.091477448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:38.361127926 +0000 UTC m=+22.704063005" watchObservedRunningTime="2026-07-15 13:29:38.361136489 +0000 UTC m=+22.704071569"
Jul 15 13:29:38.409239 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.409191    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bk9wk" podStartSLOduration=2.946320076 podStartE2EDuration="22.409175646s" totalImagesPullingTime="19.46285557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:38.408783613 +0000 UTC m=+22.751718693" watchObservedRunningTime="2026-07-15 13:29:38.409175646 +0000 UTC m=+22.752110726"
Jul 15 13:29:38.429545 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.429502    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w7dvt" podStartSLOduration=2.997482824 podStartE2EDuration="22.429487508s" totalImagesPullingTime="19.432004684s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:38.429434929 +0000 UTC m=+22.772370008" watchObservedRunningTime="2026-07-15 13:29:38.429487508 +0000 UTC m=+22.772422588"
Jul 15 13:29:38.563126 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.562937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:38.640757 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.640726    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:38.713103 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:38.713069    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:39.262293 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:39.262251    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:29:39.262728 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:39.262663    2577 generic.go:356] "Generic (PLEG): container finished" podID="8062b1d7-6402-4374-ae45-de07b61247a3" containerID="f62a8e2129aa8c1944e40e999ae68e71165b20f91715af1e87db0852a06e7d86" exitCode=1
Jul 15 13:29:39.262826 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:39.262806    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerDied","Data":"f62a8e2129aa8c1944e40e999ae68e71165b20f91715af1e87db0852a06e7d86"}
Jul 15 13:29:39.262935 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:39.262919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerStarted","Data":"4f5ac6c8a27aa69f87fd16636d30950dbc8d8de04c6971b79229fbbaa8d75e1a"}
Jul 15 13:29:39.262987 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:39.262973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerStarted","Data":"d28040e3b71b1640512a7421988a4f798069e0c0979293095a2ddab315c13bdf"}
Jul 15 13:29:39.263039 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:39.263026    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerStarted","Data":"9af3b80a2e643ebe0ea5f12929d48f54ec75bb4f7492fbea58478775f7e5f40b"}
Jul 15 13:29:39.263082 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:39.263075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerStarted","Data":"f861e288078764b00cc8d894d50c2a83dad7e583fbcc08e87177418b630b1c9a"}
Jul 15 13:29:39.278131 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:39.278104    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 13:29:39.594096 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:39.594018    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:39.594584 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:39.594568    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:40.103706 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.103586    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T13:29:39.278125651Z","UUID":"1aa2c1f9-ca00-42be-90e0-f09eba6d6682","Handler":null,"Name":"","Endpoint":""}
Jul 15 13:29:40.105498 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.105469    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 13:29:40.105618 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.105515    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 13:29:40.140875 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.140844    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:40.141041 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:40.140952    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:40.141216 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.141202    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:40.141359 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:40.141281    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:40.141359 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.141342    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:40.141466 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:40.141395    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:40.267404 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.267365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q" event={"ID":"84c6a2f9-a280-4f19-bb68-4699778508ca","Type":"ContainerStarted","Data":"94ce1ebd86c794e430c4952105719179c7324fe016e6a1b54e569a337e1e18ca"}
Jul 15 13:29:40.272432 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.272397    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-djvv5" event={"ID":"a8ed5e0f-ce91-4abf-afc9-e5425a797d09","Type":"ContainerStarted","Data":"32d237c330ec86b8382c5599eea3116b24677b77b73bb5bcae7014bc9b997a11"}
Jul 15 13:29:40.272579 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.272457    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:40.272756 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.272733    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6mkwn"
Jul 15 13:29:40.294642 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.294588    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-djvv5" podStartSLOduration=5.219126763 podStartE2EDuration="24.294562937s" totalImagesPullingTime="19.075436174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:40.294442745 +0000 UTC m=+24.637377851" watchObservedRunningTime="2026-07-15 13:29:40.294562937 +0000 UTC m=+24.637498019"
Jul 15 13:29:40.444304 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.444202    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:40.540688 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.540605    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:40.639714 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.639679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:40.871015 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.870766    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:40.966643 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:40.966616    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:41.070406 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:41.070329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:41.279514 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:41.279486    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:29:41.279953 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:41.279904    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerStarted","Data":"ff121f04f87a67ef1ac4d93004a5cf18a2ef8abf9ab20e7229a46ea5453ddacc"}
Jul 15 13:29:41.282108 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:41.282076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q" event={"ID":"84c6a2f9-a280-4f19-bb68-4699778508ca","Type":"ContainerStarted","Data":"71d3dcef3416b87ffbe518515fe09effad06e40feee5edae4816ae210ca59e41"}
Jul 15 13:29:41.304514 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:41.304448    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67v6q" podStartSLOduration=3.490415535 podStartE2EDuration="25.304428345s" totalImagesPullingTime="21.81401281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:41.30395179 +0000 UTC m=+25.646886871" watchObservedRunningTime="2026-07-15 13:29:41.304428345 +0000 UTC m=+25.647363429"
Jul 15 13:29:41.380363 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:41.380249    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:41.485774 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:41.485731    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:41.774514 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:41.774478    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:41.845545 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:41.845515    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:42.060849 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:42.060695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:42.141047 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:42.141016    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:42.141047 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:42.141052    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:42.141295 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:42.141016    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:42.141295 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:42.141157    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:42.141295 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:42.141279    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:42.141453 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:42.141383    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:42.161065 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:42.161029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:42.574686 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:42.574659    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:42.976895 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:42.976861    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:43.077423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:43.077231    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:43.289470 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:43.289447    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:29:43.289840 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:43.289817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerStarted","Data":"83914612b26a2548404de10b36292dbdca9e6ad8c83ec9fc0694ebbca05fdd15"}
Jul 15 13:29:43.290117 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:43.290078    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:43.290252 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:43.290148    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:43.290347 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:43.290332    2577 scope.go:117] "RemoveContainer" containerID="f62a8e2129aa8c1944e40e999ae68e71165b20f91715af1e87db0852a06e7d86"
Jul 15 13:29:43.291853 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:43.291825    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c1ad8c2-e440-496a-b387-7fccfdfd5221" containerID="7e36abd8c5d0241e731c84e2fba1a19ecf84f17de0871b09e71d1240b5a3c816" exitCode=0
Jul 15 13:29:43.291937 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:43.291877    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6kv7" event={"ID":"5c1ad8c2-e440-496a-b387-7fccfdfd5221","Type":"ContainerDied","Data":"7e36abd8c5d0241e731c84e2fba1a19ecf84f17de0871b09e71d1240b5a3c816"}
Jul 15 13:29:43.307995 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:43.307974    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:44.141304 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:44.141246    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:44.141955 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:44.141385    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:44.141955 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:44.141285    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:44.141955 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:44.141451    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:44.141955 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:44.141246    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:44.141955 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:44.141506    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:44.282111 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:44.282078    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:44.297640 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:44.297613    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:29:44.297987 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:44.297965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" event={"ID":"8062b1d7-6402-4374-ae45-de07b61247a3","Type":"ContainerStarted","Data":"152ff979c775fd72d1b6327f3f89161df3026a06b146b21f70b42270873ba0b5"}
Jul 15 13:29:44.298155 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:44.298136    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:44.313966 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:44.313942    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:29:44.331583 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:44.331531    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml" podStartSLOduration=8.811082717 podStartE2EDuration="28.331514728s" totalImagesPullingTime="19.520432011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:44.329406482 +0000 UTC m=+28.672341580" watchObservedRunningTime="2026-07-15 13:29:44.331514728 +0000 UTC m=+28.674449809"
Jul 15 13:29:45.014710 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:45.014676    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b8ddh"]
Jul 15 13:29:45.014879 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:45.014817    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:45.014984 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:45.014950    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:45.018310 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:45.018283    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-548bh"]
Jul 15 13:29:45.018418 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:45.018387    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:45.018509 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:45.018488    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:45.018878 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:45.018860    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xzbd5"]
Jul 15 13:29:45.018969 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:45.018937    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:45.019041 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:45.019018    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:45.302516 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:45.302482    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c1ad8c2-e440-496a-b387-7fccfdfd5221" containerID="45904467cb4de5f3afaf741bb1d9e9a9ad2b5b41ee8483b761d9f995d2a6c6d0" exitCode=0
Jul 15 13:29:45.302914 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:45.302572    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6kv7" event={"ID":"5c1ad8c2-e440-496a-b387-7fccfdfd5221","Type":"ContainerDied","Data":"45904467cb4de5f3afaf741bb1d9e9a9ad2b5b41ee8483b761d9f995d2a6c6d0"}
Jul 15 13:29:46.144406 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:46.144381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:46.144554 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:46.144471    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:46.144554 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:46.144522    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:46.144634 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:46.144616    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:46.957812 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:46.957777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:47.042831 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:47.042802    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:47.141429 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:47.141397    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:47.141580 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:47.141497    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:47.310413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:47.310382    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c1ad8c2-e440-496a-b387-7fccfdfd5221" containerID="4751252d56b549e93dc8288eb93f2e9db781cb33b9b3b303d707f0fbfc7a46c8" exitCode=0
Jul 15 13:29:47.310569 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:47.310436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6kv7" event={"ID":"5c1ad8c2-e440-496a-b387-7fccfdfd5221","Type":"ContainerDied","Data":"4751252d56b549e93dc8288eb93f2e9db781cb33b9b3b303d707f0fbfc7a46c8"}
Jul 15 13:29:47.990558 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:47.988438    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:48.141089 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:48.141052    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:48.141290 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:48.141052    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:48.141290 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:48.141187    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:48.141290 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:48.141234    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:48.398499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:48.398464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:48.504068 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:48.504028    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:48.598220 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:48.598163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:49.125873 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:49.125841    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:49.140947 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:49.140910    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:49.141125 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:49.141040    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:49.173278 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:49.173249    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gdtmh_cedb62b6-fd1f-4ccc-9b9e-987f001a3879/dns-node-resolver/0.log"
Jul 15 13:29:49.216257 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:49.216223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:49.804619 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:49.804578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:49.804795 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:49.804650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:49.804795 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:49.804676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbqc\" (UniqueName: \"kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc\") pod \"network-check-target-548bh\" (UID: \"1bfa268c-edc9-4417-a838-d777e9b9e940\") " pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:49.804795 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:49.804728    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:49.804925 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:49.804800    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:49.804925 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:49.804804    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 13:29:49.804925 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:49.804879    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 13:29:49.804925 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:49.804893    2577 projected.go:196] Error preparing data for projected volume kube-api-access-lpbqc for pod openshift-network-diagnostics/network-check-target-548bh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:49.804925 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:49.804810    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret podName:e918381b-1b9b-414a-80cb-a2aeac624753 nodeName:}" failed. No retries permitted until 2026-07-15 13:30:21.804787277 +0000 UTC m=+66.147722355 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret") pod "global-pull-secret-syncer-xzbd5" (UID: "e918381b-1b9b-414a-80cb-a2aeac624753") : object "kube-system"/"original-pull-secret" not registered
Jul 15 13:29:49.805079 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:49.804949    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs podName:7ce063d2-56c7-4d5b-b506-efee6758d32c nodeName:}" failed. No retries permitted until 2026-07-15 13:30:21.804933567 +0000 UTC m=+66.147868636 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs") pod "network-metrics-daemon-b8ddh" (UID: "7ce063d2-56c7-4d5b-b506-efee6758d32c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 13:29:49.805079 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:49.804967    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc podName:1bfa268c-edc9-4417-a838-d777e9b9e940 nodeName:}" failed. No retries permitted until 2026-07-15 13:30:21.804961205 +0000 UTC m=+66.147896263 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpbqc" (UniqueName: "kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc") pod "network-check-target-548bh" (UID: "1bfa268c-edc9-4417-a838-d777e9b9e940") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 13:29:50.144886 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:50.144855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:50.145366 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:50.144984    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b8ddh" podUID="7ce063d2-56c7-4d5b-b506-efee6758d32c"
Jul 15 13:29:50.145366 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:50.145041    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:50.145366 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:50.145132    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-548bh" podUID="1bfa268c-edc9-4417-a838-d777e9b9e940"
Jul 15 13:29:50.373743 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:50.373710    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w7dvt_c79e9a2e-64e5-487c-b25b-07b6853db2cf/node-ca/0.log"
Jul 15 13:29:50.783950 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:50.783892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:50.883519 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:50.883481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:51.140995 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:51.140901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:51.141164 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:29:51.141019    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzbd5" podUID="e918381b-1b9b-414a-80cb-a2aeac624753"
Jul 15 13:29:51.932099 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:51.932063    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:52.011752 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.011730    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-134.ec2.internal" event="NodeReady"
Jul 15 13:29:52.011912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.011878    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 13:29:52.141507 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.141471    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:29:52.141507 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.141506    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:29:52.144839 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.144817    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 13:29:52.144839 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.144826    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 13:29:52.145098 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.145084    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 13:29:52.145150 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.145126    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rv6f8\""
Jul 15 13:29:52.145203 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.145185    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w89gv\""
Jul 15 13:29:52.324473 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.324126    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c1ad8c2-e440-496a-b387-7fccfdfd5221" containerID="93faecb2ddbc4119f7be49b48ae71e099f0a6db431cf3b7f7665880788634e6a" exitCode=0
Jul 15 13:29:52.324473 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:52.324199    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6kv7" event={"ID":"5c1ad8c2-e440-496a-b387-7fccfdfd5221","Type":"ContainerDied","Data":"93faecb2ddbc4119f7be49b48ae71e099f0a6db431cf3b7f7665880788634e6a"}
Jul 15 13:29:53.141440 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:53.141404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:29:53.144427 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:53.144405    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 13:29:53.330740 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:53.330696    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c1ad8c2-e440-496a-b387-7fccfdfd5221" containerID="2e11c64da79d9d0c6ddc1c282d812e72d354e03129fb46a4e2aa02aee5f8910b" exitCode=0
Jul 15 13:29:53.330876 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:53.330734    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6kv7" event={"ID":"5c1ad8c2-e440-496a-b387-7fccfdfd5221","Type":"ContainerDied","Data":"2e11c64da79d9d0c6ddc1c282d812e72d354e03129fb46a4e2aa02aee5f8910b"}
Jul 15 13:29:53.507511 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:53.507476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:53.620027 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:53.619973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:53.721183 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:53.721105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:53.823692 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:53.823661    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:53.922560 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:53.922522    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:54.336887 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:54.336851    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d6kv7" event={"ID":"5c1ad8c2-e440-496a-b387-7fccfdfd5221","Type":"ContainerStarted","Data":"ef3af1e7b76c7a67b38a1191db7f1b11e991ddc86293f0a9c752c413103f297b"}
Jul 15 13:29:54.409965 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:54.409927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:54.608701 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:54.608625    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:54.711866 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:54.711830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:55.811992 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:55.811957    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:55.973773 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:55.973739    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:56.060921 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:56.060887    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:56.204800 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:56.204716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:56.376837 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:56.376804    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:56.474524 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:56.474439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:56.575581 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:56.575546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:56.675614 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:56.675574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:56.776537 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:56.776510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:57.174854 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:57.174766    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:57.375126 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:57.375095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:57.783876 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:57.783841    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:58.241062 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:58.240962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:58.318721 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:58.318681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:58.388863 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:58.388830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:58.935203 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:58.935170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:59.032881 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:59.032847    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:59.099035 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:59.099004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:59.397863 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:59.397830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:59.489317 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:59.489278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:59.594337 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:59.594295    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:29:59.869303 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:29:59.869237    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:00.042053 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:00.042013    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:00.466452 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:00.466421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:00.587936 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:00.587893    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:00.682486 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:00.682455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:00.796923 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:00.796891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:01.479432 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:01.479393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:01.892797 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:01.892755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:01.997499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:01.997468    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:02.989364 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:02.989330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:03.185237 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:03.185196    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:03.390927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:03.390888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:04.151531 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:04.151498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:04.243853 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:04.243824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:04.392027 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:04.391996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:04.640597 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:04.640562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:11.081105 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:11.081076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:11.199713 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:11.199679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:12.636951 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.636899    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d6kv7" podStartSLOduration=25.807891041 podStartE2EDuration="56.636881886s" totalImagesPullingTime="30.828990845s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:29:54.379710267 +0000 UTC m=+38.722645369" watchObservedRunningTime="2026-07-15 13:30:12.636881886 +0000 UTC m=+56.979816960"
Jul 15 13:30:12.637588 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.637570    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f9757bbc6-9sdnx"]
Jul 15 13:30:12.640255 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.640236    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.643469 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.643452    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 13:30:12.643574 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.643452    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 13:30:12.643616 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.643607    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t7cjq\""
Jul 15 13:30:12.643945 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.643929    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 13:30:12.653996 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.653972    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 13:30:12.667461 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.667039    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"]
Jul 15 13:30:12.669506 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.669485    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"]
Jul 15 13:30:12.669652 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.669633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"
Jul 15 13:30:12.671234 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.671215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f9757bbc6-9sdnx"]
Jul 15 13:30:12.671330 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.671311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:12.674970 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.674949    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 15 13:30:12.675105 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.674953    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 13:30:12.675105 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.675072    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 13:30:12.675249 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.675236    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 13:30:12.675614 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.675600    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5xbwx\""
Jul 15 13:30:12.675793 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.675781    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 15 13:30:12.675996 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.675981    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"]
Jul 15 13:30:12.677847 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.677831    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.680634 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.680615    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 15 13:30:12.680814 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.680792    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 15 13:30:12.680952 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.680930    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 15 13:30:12.681016 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.680972    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 15 13:30:12.681086 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.681069    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"]
Jul 15 13:30:12.684258 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.684226    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:12.693776 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.693752    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"]
Jul 15 13:30:12.708194 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.708172    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"]
Jul 15 13:30:12.710546 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.710522    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f9757bbc6-9sdnx"]
Jul 15 13:30:12.710731 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:30:12.710708    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-qshhp registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx" podUID="7eb194be-7272-4bbd-bd55-938455137b8b"
Jul 15 13:30:12.757485 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.757448    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-4hhbg"]
Jul 15 13:30:12.759639 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.759598    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hg9g7"]
Jul 15 13:30:12.759932 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.759915    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-4hhbg"
Jul 15 13:30:12.761223 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.761205    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rdb96"]
Jul 15 13:30:12.761348 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.761334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hg9g7"
Jul 15 13:30:12.762921 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.762905    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.764029 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.763986    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 13:30:12.764174 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.764161    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 13:30:12.764231 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.764203    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wxm5t\""
Jul 15 13:30:12.766822 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.766805    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 13:30:12.766892 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.766865    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 13:30:12.767351 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.767330    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 13:30:12.767848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.767834    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 13:30:12.768050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.768036    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vgfvj\""
Jul 15 13:30:12.768195 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.768162    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 13:30:12.768305 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.768288    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 13:30:12.768372 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.768297    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qvx6p\""
Jul 15 13:30:12.768456 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.768428    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 13:30:12.769710 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a8ced6c3-4029-4619-90d8-a34bd258b145-klusterlet-config\") pod \"klusterlet-addon-workmgr-55cbf69d86-87fm4\" (UID: \"a8ced6c3-4029-4619-90d8-a34bd258b145\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:12.769804 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-registry-tls\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.769804 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-bound-sa-token\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.769804 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769782    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmcnj\" (UniqueName: \"kubernetes.io/projected/3ea51344-6f17-4d94-ae07-70ebec159044-kube-api-access-bmcnj\") pod \"managed-serviceaccount-addon-agent-577656dd74-fqrb5\" (UID: \"3ea51344-6f17-4d94-ae07-70ebec159044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"
Jul 15 13:30:12.769941 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ced6c3-4029-4619-90d8-a34bd258b145-tmp\") pod \"klusterlet-addon-workmgr-55cbf69d86-87fm4\" (UID: \"a8ced6c3-4029-4619-90d8-a34bd258b145\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:12.769941 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qshhp\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-kube-api-access-qshhp\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.769941 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6zkt\" (UniqueName: \"kubernetes.io/projected/a8ced6c3-4029-4619-90d8-a34bd258b145-kube-api-access-q6zkt\") pod \"klusterlet-addon-workmgr-55cbf69d86-87fm4\" (UID: \"a8ced6c3-4029-4619-90d8-a34bd258b145\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:12.769941 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-registry-certificates\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.769941 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769926    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ea51344-6f17-4d94-ae07-70ebec159044-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-577656dd74-fqrb5\" (UID: \"3ea51344-6f17-4d94-ae07-70ebec159044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"
Jul 15 13:30:12.770154 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eb194be-7272-4bbd-bd55-938455137b8b-ca-trust-extracted\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.770154 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.769984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-image-registry-private-configuration\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.770154 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.770014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-trusted-ca\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.770154 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.770038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-installation-pull-secrets\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.777326 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.777305    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-4hhbg"]
Jul 15 13:30:12.781916 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.781839    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hg9g7"]
Jul 15 13:30:12.786209 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.786187    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rdb96"]
Jul 15 13:30:12.797943 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.797921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:12.838216 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.838184    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-j8lb8"]
Jul 15 13:30:12.840302 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.840285    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:12.842967 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.842944    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 13:30:12.842967 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.842961    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rnzhl\""
Jul 15 13:30:12.843153 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.842961    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 13:30:12.845475 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.845455    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"]
Jul 15 13:30:12.848092 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.848073    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:12.852588 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.852566    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j8lb8"]
Jul 15 13:30:12.867536 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.867508    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"]
Jul 15 13:30:12.871128 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-registry-tls\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.871301 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871146    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/35e809cf-4d2d-4499-829f-c9e7ef493957-crio-socket\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.871301 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq4dz\" (UniqueName: \"kubernetes.io/projected/35e809cf-4d2d-4499-829f-c9e7ef493957-kube-api-access-xq4dz\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.871391 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h56kb\" (UniqueName: \"kubernetes.io/projected/6222ec40-0d4d-460c-9528-66af0c47c175-kube-api-access-h56kb\") pod \"downloads-68c9cbb497-4hhbg\" (UID: \"6222ec40-0d4d-460c-9528-66af0c47c175\") " pod="openshift-console/downloads-68c9cbb497-4hhbg"
Jul 15 13:30:12.871391 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871359    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ced6c3-4029-4619-90d8-a34bd258b145-tmp\") pod \"klusterlet-addon-workmgr-55cbf69d86-87fm4\" (UID: \"a8ced6c3-4029-4619-90d8-a34bd258b145\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:12.871391 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qshhp\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-kube-api-access-qshhp\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.871516 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871393    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l27sk\" (UniqueName: \"kubernetes.io/projected/f4cbbdc9-d133-4aca-a034-b49b8d8fe789-kube-api-access-l27sk\") pod \"ingress-canary-hg9g7\" (UID: \"f4cbbdc9-d133-4aca-a034-b49b8d8fe789\") " pod="openshift-ingress-canary/ingress-canary-hg9g7"
Jul 15 13:30:12.871516 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871422    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ea51344-6f17-4d94-ae07-70ebec159044-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-577656dd74-fqrb5\" (UID: \"3ea51344-6f17-4d94-ae07-70ebec159044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"
Jul 15 13:30:12.871516 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871447    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/35e809cf-4d2d-4499-829f-c9e7ef493957-data-volume\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.871516 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871491    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-ca\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.871705 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871532    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.871705 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-image-registry-private-configuration\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.871705 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-installation-pull-secrets\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.871705 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871643    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/35e809cf-4d2d-4499-829f-c9e7ef493957-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.871705 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/62541fa5-d283-4126-b07c-dbff723356e1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.871705 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a8ced6c3-4029-4619-90d8-a34bd258b145-klusterlet-config\") pod \"klusterlet-addon-workmgr-55cbf69d86-87fm4\" (UID: \"a8ced6c3-4029-4619-90d8-a34bd258b145\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:12.871970 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-bound-sa-token\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.871970 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871735    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8ced6c3-4029-4619-90d8-a34bd258b145-tmp\") pod \"klusterlet-addon-workmgr-55cbf69d86-87fm4\" (UID: \"a8ced6c3-4029-4619-90d8-a34bd258b145\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:12.871970 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.871970 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmz58\" (UniqueName: \"kubernetes.io/projected/62541fa5-d283-4126-b07c-dbff723356e1-kube-api-access-fmz58\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.871970 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmcnj\" (UniqueName: \"kubernetes.io/projected/3ea51344-6f17-4d94-ae07-70ebec159044-kube-api-access-bmcnj\") pod \"managed-serviceaccount-addon-agent-577656dd74-fqrb5\" (UID: \"3ea51344-6f17-4d94-ae07-70ebec159044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"
Jul 15 13:30:12.872216 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.871995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6zkt\" (UniqueName: \"kubernetes.io/projected/a8ced6c3-4029-4619-90d8-a34bd258b145-kube-api-access-q6zkt\") pod \"klusterlet-addon-workmgr-55cbf69d86-87fm4\" (UID: \"a8ced6c3-4029-4619-90d8-a34bd258b145\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:12.872216 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.872112    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eb194be-7272-4bbd-bd55-938455137b8b-ca-trust-extracted\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.872216 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.872148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-registry-certificates\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.872216 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.872194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-hub\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.872429 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.872234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-trusted-ca\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.872429 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.872283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/35e809cf-4d2d-4499-829f-c9e7ef493957-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.872429 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.872313    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4cbbdc9-d133-4aca-a034-b49b8d8fe789-cert\") pod \"ingress-canary-hg9g7\" (UID: \"f4cbbdc9-d133-4aca-a034-b49b8d8fe789\") " pod="openshift-ingress-canary/ingress-canary-hg9g7"
Jul 15 13:30:12.873169 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.872865    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eb194be-7272-4bbd-bd55-938455137b8b-ca-trust-extracted\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.873488 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.873417    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-trusted-ca\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.873488 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.873442    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-registry-certificates\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.875886 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.875862    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-image-registry-private-configuration\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.875886 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.875878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-installation-pull-secrets\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.876009 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.875972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a8ced6c3-4029-4619-90d8-a34bd258b145-klusterlet-config\") pod \"klusterlet-addon-workmgr-55cbf69d86-87fm4\" (UID: \"a8ced6c3-4029-4619-90d8-a34bd258b145\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:12.876043 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.876013    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ea51344-6f17-4d94-ae07-70ebec159044-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-577656dd74-fqrb5\" (UID: \"3ea51344-6f17-4d94-ae07-70ebec159044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"
Jul 15 13:30:12.876244 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.876228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-registry-tls\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.893823 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.893763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmcnj\" (UniqueName: \"kubernetes.io/projected/3ea51344-6f17-4d94-ae07-70ebec159044-kube-api-access-bmcnj\") pod \"managed-serviceaccount-addon-agent-577656dd74-fqrb5\" (UID: \"3ea51344-6f17-4d94-ae07-70ebec159044\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"
Jul 15 13:30:12.893927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.893866    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6zkt\" (UniqueName: \"kubernetes.io/projected/a8ced6c3-4029-4619-90d8-a34bd258b145-kube-api-access-q6zkt\") pod \"klusterlet-addon-workmgr-55cbf69d86-87fm4\" (UID: \"a8ced6c3-4029-4619-90d8-a34bd258b145\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:12.893968 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.893944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-bound-sa-token\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.894194 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.894173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qshhp\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-kube-api-access-qshhp\") pod \"image-registry-6f9757bbc6-9sdnx\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") " pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:12.927250 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.927217    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:12.973611 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.973563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs5jl\" (UniqueName: \"kubernetes.io/projected/19557185-bb6a-4cb2-97c0-71457f1f9eb8-kube-api-access-fs5jl\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:12.973611 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.973618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/522a352f-f06c-4132-968a-4c275d0cc547-registry-certificates\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:12.973846 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.973681    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/522a352f-f06c-4132-968a-4c275d0cc547-bound-sa-token\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:12.973846 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.973714    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgfq5\" (UniqueName: \"kubernetes.io/projected/522a352f-f06c-4132-968a-4c275d0cc547-kube-api-access-wgfq5\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:12.973846 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.973762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-hub\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.973846 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.973793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19557185-bb6a-4cb2-97c0-71457f1f9eb8-config-volume\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:12.973846 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.973831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/35e809cf-4d2d-4499-829f-c9e7ef493957-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.974076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.973935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4cbbdc9-d133-4aca-a034-b49b8d8fe789-cert\") pod \"ingress-canary-hg9g7\" (UID: \"f4cbbdc9-d133-4aca-a034-b49b8d8fe789\") " pod="openshift-ingress-canary/ingress-canary-hg9g7"
Jul 15 13:30:12.974076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.973975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/35e809cf-4d2d-4499-829f-c9e7ef493957-crio-socket\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.974076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq4dz\" (UniqueName: \"kubernetes.io/projected/35e809cf-4d2d-4499-829f-c9e7ef493957-kube-api-access-xq4dz\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.974076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974034    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h56kb\" (UniqueName: \"kubernetes.io/projected/6222ec40-0d4d-460c-9528-66af0c47c175-kube-api-access-h56kb\") pod \"downloads-68c9cbb497-4hhbg\" (UID: \"6222ec40-0d4d-460c-9528-66af0c47c175\") " pod="openshift-console/downloads-68c9cbb497-4hhbg"
Jul 15 13:30:12.974076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l27sk\" (UniqueName: \"kubernetes.io/projected/f4cbbdc9-d133-4aca-a034-b49b8d8fe789-kube-api-access-l27sk\") pod \"ingress-canary-hg9g7\" (UID: \"f4cbbdc9-d133-4aca-a034-b49b8d8fe789\") " pod="openshift-ingress-canary/ingress-canary-hg9g7"
Jul 15 13:30:12.974076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/35e809cf-4d2d-4499-829f-c9e7ef493957-crio-socket\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.974404 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/522a352f-f06c-4132-968a-4c275d0cc547-registry-tls\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:12.974404 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/35e809cf-4d2d-4499-829f-c9e7ef493957-data-volume\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.974404 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-ca\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.974404 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/522a352f-f06c-4132-968a-4c275d0cc547-ca-trust-extracted\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:12.974404 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.974404 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/522a352f-f06c-4132-968a-4c275d0cc547-installation-pull-secrets\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:12.974404 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19557185-bb6a-4cb2-97c0-71457f1f9eb8-tmp-dir\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:12.974786 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/35e809cf-4d2d-4499-829f-c9e7ef493957-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.974786 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/62541fa5-d283-4126-b07c-dbff723356e1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.974786 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/522a352f-f06c-4132-968a-4c275d0cc547-image-registry-private-configuration\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:12.974786 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.974786 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974539    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/35e809cf-4d2d-4499-829f-c9e7ef493957-data-volume\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.974786 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmz58\" (UniqueName: \"kubernetes.io/projected/62541fa5-d283-4126-b07c-dbff723356e1-kube-api-access-fmz58\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.974786 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19557185-bb6a-4cb2-97c0-71457f1f9eb8-metrics-tls\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:12.974786 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.974624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/522a352f-f06c-4132-968a-4c275d0cc547-trusted-ca\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:12.975312 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.975286    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/35e809cf-4d2d-4499-829f-c9e7ef493957-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.975520 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.975487    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/62541fa5-d283-4126-b07c-dbff723356e1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.977024 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.976996    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/35e809cf-4d2d-4499-829f-c9e7ef493957-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.977130 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.977066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-hub\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.977318 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.977296    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.977410 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.977329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4cbbdc9-d133-4aca-a034-b49b8d8fe789-cert\") pod \"ingress-canary-hg9g7\" (UID: \"f4cbbdc9-d133-4aca-a034-b49b8d8fe789\") " pod="openshift-ingress-canary/ingress-canary-hg9g7"
Jul 15 13:30:12.977696 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.977677    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.977919 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.977897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/62541fa5-d283-4126-b07c-dbff723356e1-ca\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:12.995219 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.995189    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h56kb\" (UniqueName: \"kubernetes.io/projected/6222ec40-0d4d-460c-9528-66af0c47c175-kube-api-access-h56kb\") pod \"downloads-68c9cbb497-4hhbg\" (UID: \"6222ec40-0d4d-460c-9528-66af0c47c175\") " pod="openshift-console/downloads-68c9cbb497-4hhbg"
Jul 15 13:30:12.995318 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.995227    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l27sk\" (UniqueName: \"kubernetes.io/projected/f4cbbdc9-d133-4aca-a034-b49b8d8fe789-kube-api-access-l27sk\") pod \"ingress-canary-hg9g7\" (UID: \"f4cbbdc9-d133-4aca-a034-b49b8d8fe789\") " pod="openshift-ingress-canary/ingress-canary-hg9g7"
Jul 15 13:30:12.995365 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.995340    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq4dz\" (UniqueName: \"kubernetes.io/projected/35e809cf-4d2d-4499-829f-c9e7ef493957-kube-api-access-xq4dz\") pod \"insights-runtime-extractor-rdb96\" (UID: \"35e809cf-4d2d-4499-829f-c9e7ef493957\") " pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:12.996502 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:12.996486    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmz58\" (UniqueName: \"kubernetes.io/projected/62541fa5-d283-4126-b07c-dbff723356e1-kube-api-access-fmz58\") pod \"cluster-proxy-proxy-agent-f8d9746d9-5zm67\" (UID: \"62541fa5-d283-4126-b07c-dbff723356e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:13.002316 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.002297    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"
Jul 15 13:30:13.009025 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.009007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:13.015196 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.015171    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"
Jul 15 13:30:13.025871 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.025833    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/522a352f-f06c-4132-968a-4c275d0cc547-registry-tls\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/522a352f-f06c-4132-968a-4c275d0cc547-ca-trust-extracted\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/522a352f-f06c-4132-968a-4c275d0cc547-installation-pull-secrets\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19557185-bb6a-4cb2-97c0-71457f1f9eb8-tmp-dir\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/522a352f-f06c-4132-968a-4c275d0cc547-image-registry-private-configuration\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075620    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19557185-bb6a-4cb2-97c0-71457f1f9eb8-metrics-tls\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/522a352f-f06c-4132-968a-4c275d0cc547-trusted-ca\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs5jl\" (UniqueName: \"kubernetes.io/projected/19557185-bb6a-4cb2-97c0-71457f1f9eb8-kube-api-access-fs5jl\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/522a352f-f06c-4132-968a-4c275d0cc547-registry-certificates\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/522a352f-f06c-4132-968a-4c275d0cc547-bound-sa-token\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgfq5\" (UniqueName: \"kubernetes.io/projected/522a352f-f06c-4132-968a-4c275d0cc547-kube-api-access-wgfq5\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.075788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19557185-bb6a-4cb2-97c0-71457f1f9eb8-config-volume\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:13.076912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.076443    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19557185-bb6a-4cb2-97c0-71457f1f9eb8-config-volume\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:13.080413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.080383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19557185-bb6a-4cb2-97c0-71457f1f9eb8-tmp-dir\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:13.081422 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.081236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/522a352f-f06c-4132-968a-4c275d0cc547-trusted-ca\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.081422 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.081349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/522a352f-f06c-4132-968a-4c275d0cc547-ca-trust-extracted\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.082144 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.082117    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/522a352f-f06c-4132-968a-4c275d0cc547-registry-certificates\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.084666 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.084461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-4hhbg"
Jul 15 13:30:13.091202 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.091027    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19557185-bb6a-4cb2-97c0-71457f1f9eb8-metrics-tls\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:13.091202 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.091146    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hg9g7"
Jul 15 13:30:13.092495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.092074    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/522a352f-f06c-4132-968a-4c275d0cc547-registry-tls\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.092495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.092078    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/522a352f-f06c-4132-968a-4c275d0cc547-image-registry-private-configuration\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.096069 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.095884    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rdb96"
Jul 15 13:30:13.101969 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.101909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgfq5\" (UniqueName: \"kubernetes.io/projected/522a352f-f06c-4132-968a-4c275d0cc547-kube-api-access-wgfq5\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.102701 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.102675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/522a352f-f06c-4132-968a-4c275d0cc547-installation-pull-secrets\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.103194 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.103171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/522a352f-f06c-4132-968a-4c275d0cc547-bound-sa-token\") pod \"image-registry-5dd7fddbc6-bd6xz\" (UID: \"522a352f-f06c-4132-968a-4c275d0cc547\") " pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.105431 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.105250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs5jl\" (UniqueName: \"kubernetes.io/projected/19557185-bb6a-4cb2-97c0-71457f1f9eb8-kube-api-access-fs5jl\") pod \"dns-default-j8lb8\" (UID: \"19557185-bb6a-4cb2-97c0-71457f1f9eb8\") " pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:13.121503 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.120695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:13.151660 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.151107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:13.159433 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.158771    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:13.215982 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.214470    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5"]
Jul 15 13:30:13.251810 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.250242    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"]
Jul 15 13:30:13.256345 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.256312    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67"]
Jul 15 13:30:13.271024 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:30:13.270955    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ced6c3_4029_4619_90d8_a34bd258b145.slice/crio-1f7ee62cbb7e425259217b4faab79cc984cf19bb2c4e90dc2870f9dd2a27d316 WatchSource:0}: Error finding container 1f7ee62cbb7e425259217b4faab79cc984cf19bb2c4e90dc2870f9dd2a27d316: Status 404 returned error can't find the container with id 1f7ee62cbb7e425259217b4faab79cc984cf19bb2c4e90dc2870f9dd2a27d316
Jul 15 13:30:13.271567 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:30:13.271519    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62541fa5_d283_4126_b07c_dbff723356e1.slice/crio-3094ebf156e241c1d64089ad3bcb2c68ac72c60a6f34be7fb962069dacf80759 WatchSource:0}: Error finding container 3094ebf156e241c1d64089ad3bcb2c68ac72c60a6f34be7fb962069dacf80759: Status 404 returned error can't find the container with id 3094ebf156e241c1d64089ad3bcb2c68ac72c60a6f34be7fb962069dacf80759
Jul 15 13:30:13.327895 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.327864    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rdb96"]
Jul 15 13:30:13.330664 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:30:13.330631    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e809cf_4d2d_4499_829f_c9e7ef493957.slice/crio-5dd63570d97739f8a3e480caadb0bd0237f49dd9702fc71383bd9caec02ce272 WatchSource:0}: Error finding container 5dd63570d97739f8a3e480caadb0bd0237f49dd9702fc71383bd9caec02ce272: Status 404 returned error can't find the container with id 5dd63570d97739f8a3e480caadb0bd0237f49dd9702fc71383bd9caec02ce272
Jul 15 13:30:13.346409 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.346333    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hg9g7"]
Jul 15 13:30:13.350552 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:30:13.350526    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4cbbdc9_d133_4aca_a034_b49b8d8fe789.slice/crio-73bac5a5a5bc4a6b87e70a77eb6115056b9b7dbf0d5d83cc7027c2b60e2009fd WatchSource:0}: Error finding container 73bac5a5a5bc4a6b87e70a77eb6115056b9b7dbf0d5d83cc7027c2b60e2009fd: Status 404 returned error can't find the container with id 73bac5a5a5bc4a6b87e70a77eb6115056b9b7dbf0d5d83cc7027c2b60e2009fd
Jul 15 13:30:13.355301 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.355249    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-4hhbg"]
Jul 15 13:30:13.360690 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:30:13.360664    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6222ec40_0d4d_460c_9528_66af0c47c175.slice/crio-8fcb3c2cf7066d16a3e957d04c191af17ddd093038458ba6cc7d7f6efb2ae4c8 WatchSource:0}: Error finding container 8fcb3c2cf7066d16a3e957d04c191af17ddd093038458ba6cc7d7f6efb2ae4c8: Status 404 returned error can't find the container with id 8fcb3c2cf7066d16a3e957d04c191af17ddd093038458ba6cc7d7f6efb2ae4c8
Jul 15 13:30:13.375154 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.375117    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j8lb8"]
Jul 15 13:30:13.378175 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:30:13.378146    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19557185_bb6a_4cb2_97c0_71457f1f9eb8.slice/crio-31dfc1cd6ee038ae890da7d1dc931e06283ac0749fea2406645b3c2d61618f64 WatchSource:0}: Error finding container 31dfc1cd6ee038ae890da7d1dc931e06283ac0749fea2406645b3c2d61618f64: Status 404 returned error can't find the container with id 31dfc1cd6ee038ae890da7d1dc931e06283ac0749fea2406645b3c2d61618f64
Jul 15 13:30:13.383390 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.383364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j8lb8" event={"ID":"19557185-bb6a-4cb2-97c0-71457f1f9eb8","Type":"ContainerStarted","Data":"31dfc1cd6ee038ae890da7d1dc931e06283ac0749fea2406645b3c2d61618f64"}
Jul 15 13:30:13.384479 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.384460    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-4hhbg" event={"ID":"6222ec40-0d4d-460c-9528-66af0c47c175","Type":"ContainerStarted","Data":"8fcb3c2cf7066d16a3e957d04c191af17ddd093038458ba6cc7d7f6efb2ae4c8"}
Jul 15 13:30:13.385527 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.385508    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hg9g7" event={"ID":"f4cbbdc9-d133-4aca-a034-b49b8d8fe789","Type":"ContainerStarted","Data":"73bac5a5a5bc4a6b87e70a77eb6115056b9b7dbf0d5d83cc7027c2b60e2009fd"}
Jul 15 13:30:13.386614 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.386586    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4" event={"ID":"a8ced6c3-4029-4619-90d8-a34bd258b145","Type":"ContainerStarted","Data":"1f7ee62cbb7e425259217b4faab79cc984cf19bb2c4e90dc2870f9dd2a27d316"}
Jul 15 13:30:13.387612 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.387586    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5" event={"ID":"3ea51344-6f17-4d94-ae07-70ebec159044","Type":"ContainerStarted","Data":"d543abd6e5e9227799c02a8307159e0e7e834a18c07165161ff07db0f85174be"}
Jul 15 13:30:13.388515 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.388497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rdb96" event={"ID":"35e809cf-4d2d-4499-829f-c9e7ef493957","Type":"ContainerStarted","Data":"5dd63570d97739f8a3e480caadb0bd0237f49dd9702fc71383bd9caec02ce272"}
Jul 15 13:30:13.389474 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.389457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:13.389596 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.389474    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67" event={"ID":"62541fa5-d283-4126-b07c-dbff723356e1","Type":"ContainerStarted","Data":"3094ebf156e241c1d64089ad3bcb2c68ac72c60a6f34be7fb962069dacf80759"}
Jul 15 13:30:13.393536 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.393506    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"]
Jul 15 13:30:13.399534 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:30:13.399499    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522a352f_f06c_4132_968a_4c275d0cc547.slice/crio-bd50bb3821cbf88df6383b8e8c592f8ccf5a28aa8d50f99eb14354ec5b6c6fdb WatchSource:0}: Error finding container bd50bb3821cbf88df6383b8e8c592f8ccf5a28aa8d50f99eb14354ec5b6c6fdb: Status 404 returned error can't find the container with id bd50bb3821cbf88df6383b8e8c592f8ccf5a28aa8d50f99eb14354ec5b6c6fdb
Jul 15 13:30:13.399852 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.399834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:13.578775 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.578735    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-registry-tls\") pod \"7eb194be-7272-4bbd-bd55-938455137b8b\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") "
Jul 15 13:30:13.578975 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.578788    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-trusted-ca\") pod \"7eb194be-7272-4bbd-bd55-938455137b8b\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") "
Jul 15 13:30:13.578975 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.578809    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-registry-certificates\") pod \"7eb194be-7272-4bbd-bd55-938455137b8b\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") "
Jul 15 13:30:13.578975 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.578838    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-installation-pull-secrets\") pod \"7eb194be-7272-4bbd-bd55-938455137b8b\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") "
Jul 15 13:30:13.578975 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.578945    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eb194be-7272-4bbd-bd55-938455137b8b-ca-trust-extracted\") pod \"7eb194be-7272-4bbd-bd55-938455137b8b\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") "
Jul 15 13:30:13.579178 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.578995    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qshhp\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-kube-api-access-qshhp\") pod \"7eb194be-7272-4bbd-bd55-938455137b8b\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") "
Jul 15 13:30:13.579178 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.579043    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-image-registry-private-configuration\") pod \"7eb194be-7272-4bbd-bd55-938455137b8b\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") "
Jul 15 13:30:13.579178 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.579067    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-bound-sa-token\") pod \"7eb194be-7272-4bbd-bd55-938455137b8b\" (UID: \"7eb194be-7272-4bbd-bd55-938455137b8b\") "
Jul 15 13:30:13.579417 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.579391    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb194be-7272-4bbd-bd55-938455137b8b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7eb194be-7272-4bbd-bd55-938455137b8b" (UID: "7eb194be-7272-4bbd-bd55-938455137b8b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:30:13.579505 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.579487    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eb194be-7272-4bbd-bd55-938455137b8b-ca-trust-extracted\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:30:13.579734 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.579674    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7eb194be-7272-4bbd-bd55-938455137b8b" (UID: "7eb194be-7272-4bbd-bd55-938455137b8b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:30:13.579734 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.579704    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7eb194be-7272-4bbd-bd55-938455137b8b" (UID: "7eb194be-7272-4bbd-bd55-938455137b8b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:30:13.581392 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.581361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7eb194be-7272-4bbd-bd55-938455137b8b" (UID: "7eb194be-7272-4bbd-bd55-938455137b8b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:30:13.581514 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.581395    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7eb194be-7272-4bbd-bd55-938455137b8b" (UID: "7eb194be-7272-4bbd-bd55-938455137b8b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:30:13.581658 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.581632    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7eb194be-7272-4bbd-bd55-938455137b8b" (UID: "7eb194be-7272-4bbd-bd55-938455137b8b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:30:13.581754 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.581688    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-kube-api-access-qshhp" (OuterVolumeSpecName: "kube-api-access-qshhp") pod "7eb194be-7272-4bbd-bd55-938455137b8b" (UID: "7eb194be-7272-4bbd-bd55-938455137b8b"). InnerVolumeSpecName "kube-api-access-qshhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:30:13.581843 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.581825    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7eb194be-7272-4bbd-bd55-938455137b8b" (UID: "7eb194be-7272-4bbd-bd55-938455137b8b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:30:13.680076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.679985    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-trusted-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:30:13.680076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.680019    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eb194be-7272-4bbd-bd55-938455137b8b-registry-certificates\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:30:13.680076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.680036    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-installation-pull-secrets\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:30:13.680076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.680049    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qshhp\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-kube-api-access-qshhp\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:30:13.680076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.680062    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eb194be-7272-4bbd-bd55-938455137b8b-image-registry-private-configuration\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:30:13.680076 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.680075    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-bound-sa-token\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:30:13.680702 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:13.680088    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eb194be-7272-4bbd-bd55-938455137b8b-registry-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:30:14.230146 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:14.230028    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:14.323558 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:14.323445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:14.401854 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:14.401820    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f9757bbc6-9sdnx"
Jul 15 13:30:14.402895 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:14.402859    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz" event={"ID":"522a352f-f06c-4132-968a-4c275d0cc547","Type":"ContainerStarted","Data":"d9b3ccbb7b660608b65414e539e405142b1dfb4b64029eee5e20603bcc58fd8e"}
Jul 15 13:30:14.403025 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:14.402920    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz" event={"ID":"522a352f-f06c-4132-968a-4c275d0cc547","Type":"ContainerStarted","Data":"bd50bb3821cbf88df6383b8e8c592f8ccf5a28aa8d50f99eb14354ec5b6c6fdb"}
Jul 15 13:30:14.403158 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:14.403139    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:14.414258 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:14.414151    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:14.446911 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:14.446779    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f9757bbc6-9sdnx"]
Jul 15 13:30:14.448295 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:14.448254    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f9757bbc6-9sdnx"]
Jul 15 13:30:14.476441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:14.476354    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz" podStartSLOduration=2.476332999 podStartE2EDuration="2.476332999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:14.474926546 +0000 UTC m=+58.817861627" watchObservedRunningTime="2026-07-15 13:30:14.476332999 +0000 UTC m=+58.819268064"
Jul 15 13:30:15.753586 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:15.753550    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:15.848392 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:15.848319    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:16.151045 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:16.150759    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eb194be-7272-4bbd-bd55-938455137b8b" path="/var/lib/kubelet/pods/7eb194be-7272-4bbd-bd55-938455137b8b/volumes"
Jul 15 13:30:16.338927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:16.337238    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jvcml"
Jul 15 13:30:16.636560 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:16.636406    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:16.735481 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:16.735035    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:16.854022 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:16.853975    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:17.026075 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:17.025991    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:17.183333 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:17.183289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:17.307619 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:17.307538    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:19.160554 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:19.160519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:19.276848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:19.276812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:21.856138 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.856093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:30:21.856659 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.856170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:30:21.856659 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.856204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbqc\" (UniqueName: \"kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc\") pod \"network-check-target-548bh\" (UID: \"1bfa268c-edc9-4417-a838-d777e9b9e940\") " pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:30:21.861863 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.861831    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 13:30:21.862948 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.862921    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 13:30:21.863083 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.862921    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 13:30:21.869693 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.869666    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 13:30:21.869806 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.869728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ce063d2-56c7-4d5b-b506-efee6758d32c-metrics-certs\") pod \"network-metrics-daemon-b8ddh\" (UID: \"7ce063d2-56c7-4d5b-b506-efee6758d32c\") " pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:30:21.869806 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.869760    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e918381b-1b9b-414a-80cb-a2aeac624753-original-pull-secret\") pod \"global-pull-secret-syncer-xzbd5\" (UID: \"e918381b-1b9b-414a-80cb-a2aeac624753\") " pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:30:21.880035 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.880008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpbqc\" (UniqueName: \"kubernetes.io/projected/1bfa268c-edc9-4417-a838-d777e9b9e940-kube-api-access-lpbqc\") pod \"network-check-target-548bh\" (UID: \"1bfa268c-edc9-4417-a838-d777e9b9e940\") " pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:30:21.949937 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:21.949890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzbd5"
Jul 15 13:30:22.156146 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:22.156118    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w89gv\""
Jul 15 13:30:22.160363 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:22.160328    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rv6f8\""
Jul 15 13:30:22.163400 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:22.163378    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b8ddh"
Jul 15 13:30:22.166556 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:22.166536    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:30:23.828467 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:23.828423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:25.017628 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:25.017592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:25.104929 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:25.104880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:25.371661 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:25.371575    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:26.000455 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.000418    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:26.000638 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.000548    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xzbd5"]
Jul 15 13:30:26.090206 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.090099    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:26.271918 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:30:26.271757    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode918381b_1b9b_414a_80cb_a2aeac624753.slice/crio-45561fd4d5114c9add468417a7add9c2f54b17466a70a83fed663829b18e668b WatchSource:0}: Error finding container 45561fd4d5114c9add468417a7add9c2f54b17466a70a83fed663829b18e668b: Status 404 returned error can't find the container with id 45561fd4d5114c9add468417a7add9c2f54b17466a70a83fed663829b18e668b
Jul 15 13:30:26.472896 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.472673    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67" event={"ID":"62541fa5-d283-4126-b07c-dbff723356e1","Type":"ContainerStarted","Data":"a7963faed3a923c108abc828c78b18940dfd45ac40de214c05b4f68d8cf451b9"}
Jul 15 13:30:26.474201 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.474164    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b8ddh"]
Jul 15 13:30:26.499993 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.499904    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-548bh"]
Jul 15 13:30:26.507853 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.506628    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j8lb8" event={"ID":"19557185-bb6a-4cb2-97c0-71457f1f9eb8","Type":"ContainerStarted","Data":"ab5eddfc68985ec1383e6bee658dcdc759b6ae321db583d25a204bb22d6928a5"}
Jul 15 13:30:26.524131 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:30:26.523169    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bfa268c_edc9_4417_a838_d777e9b9e940.slice/crio-346edb6c8761be27982175c1207b718973b190864b860876a82f08625f3f40a2 WatchSource:0}: Error finding container 346edb6c8761be27982175c1207b718973b190864b860876a82f08625f3f40a2: Status 404 returned error can't find the container with id 346edb6c8761be27982175c1207b718973b190864b860876a82f08625f3f40a2
Jul 15 13:30:26.543321 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.541430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4" event={"ID":"a8ced6c3-4029-4619-90d8-a34bd258b145","Type":"ContainerStarted","Data":"1e38e9b0764bb7cbf30e7404c8fd303c6809a7b1f9774570808beb0b1b30fb9a"}
Jul 15 13:30:26.543321 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.543294    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:26.544176 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.544153    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4"
Jul 15 13:30:26.552621 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.552586    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xzbd5" event={"ID":"e918381b-1b9b-414a-80cb-a2aeac624753","Type":"ContainerStarted","Data":"45561fd4d5114c9add468417a7add9c2f54b17466a70a83fed663829b18e668b"}
Jul 15 13:30:26.566282 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.565868    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hg9g7" podStartSLOduration=1.652570825 podStartE2EDuration="14.565845498s" totalImagesPullingTime="12.913274673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:26.565087542 +0000 UTC m=+70.908022640" watchObservedRunningTime="2026-07-15 13:30:26.565845498 +0000 UTC m=+70.908780594"
Jul 15 13:30:26.609655 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:26.609402    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55cbf69d86-87fm4" podStartSLOduration=1.6090327819999999 podStartE2EDuration="14.609383506s" totalImagesPullingTime="13.000350724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:26.609172615 +0000 UTC m=+70.952107711" watchObservedRunningTime="2026-07-15 13:30:26.609383506 +0000 UTC m=+70.952318589"
Jul 15 13:30:27.567501 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.567363    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rdb96" event={"ID":"35e809cf-4d2d-4499-829f-c9e7ef493957","Type":"ContainerStarted","Data":"cee37f8cce7e24c0e5381ddd2b29b634fda5908cff32affe6d1dd27fc6ea3ee6"}
Jul 15 13:30:27.571325 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.571291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67" event={"ID":"62541fa5-d283-4126-b07c-dbff723356e1","Type":"ContainerStarted","Data":"ea65bcc2c4da150b0bcd23d3a1f5bc945f70599bb6f5c58e4e698568cbf6552e"}
Jul 15 13:30:27.571474 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.571343    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67" event={"ID":"62541fa5-d283-4126-b07c-dbff723356e1","Type":"ContainerStarted","Data":"208454f12ccb25dc4e4ee3a449e62825c7f0a5d4ca611ac3cea1061d42d8bc54"}
Jul 15 13:30:27.575113 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.574986    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-548bh" event={"ID":"1bfa268c-edc9-4417-a838-d777e9b9e940","Type":"ContainerStarted","Data":"346edb6c8761be27982175c1207b718973b190864b860876a82f08625f3f40a2"}
Jul 15 13:30:27.582617 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.581943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j8lb8" event={"ID":"19557185-bb6a-4cb2-97c0-71457f1f9eb8","Type":"ContainerStarted","Data":"284f9091431bad203cc77875b2ea63597ebfa8855a64ebb64d3681b4aa1f833d"}
Jul 15 13:30:27.582617 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.582560    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:27.585790 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.585301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hg9g7" event={"ID":"f4cbbdc9-d133-4aca-a034-b49b8d8fe789","Type":"ContainerStarted","Data":"b5ccf515f0d830e1957254fa8b372069ca92e93c61e1416d49b4f86408f9d883"}
Jul 15 13:30:27.588446 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.587987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5" event={"ID":"3ea51344-6f17-4d94-ae07-70ebec159044","Type":"ContainerStarted","Data":"9a9fa241f0d4951a99d7dc6ab6b0b0c6ef196196a44d000910ebd6be8cf7f712"}
Jul 15 13:30:27.594428 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.594373    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b8ddh" event={"ID":"7ce063d2-56c7-4d5b-b506-efee6758d32c","Type":"ContainerStarted","Data":"71ceee4b1d9f46e183bdc22e11ba4e346efa179c4a5c9b4926e9123471194a5f"}
Jul 15 13:30:27.636847 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.636786    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-577656dd74-fqrb5" podStartSLOduration=3.035406681 podStartE2EDuration="15.636762867s" totalImagesPullingTime="12.601356186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:27.636633175 +0000 UTC m=+71.979568257" watchObservedRunningTime="2026-07-15 13:30:27.636762867 +0000 UTC m=+71.979697928"
Jul 15 13:30:27.637551 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:27.637509    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8d9746d9-5zm67" podStartSLOduration=3.080024484 podStartE2EDuration="15.637491542s" totalImagesPullingTime="12.557467058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:27.60251523 +0000 UTC m=+71.945450312" watchObservedRunningTime="2026-07-15 13:30:27.637491542 +0000 UTC m=+71.980426623"
Jul 15 13:30:30.810406 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:30.810376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:31.641817 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:31.641771    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:31.739826 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:31.739785    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:32.280317 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:32.280282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:32.381802 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:32.381768    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:33.164385 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.164342    2577 patch_prober.go:28] interesting pod/image-registry-5dd7fddbc6-bd6xz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 15 13:30:33.164563 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.164411    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz" podUID="522a352f-f06c-4132-968a-4c275d0cc547" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 13:30:33.259809 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.259725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:33.352331 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.352292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:33.472413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.472323    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:33.691496 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.691464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:33.742708 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.742595    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-j8lb8" podStartSLOduration=9.637838821999999 podStartE2EDuration="21.742574763s" totalImagesPullingTime="12.104735941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:27.680061175 +0000 UTC m=+72.022996273" watchObservedRunningTime="2026-07-15 13:30:33.742574763 +0000 UTC m=+78.085509820"
Jul 15 13:30:33.743356 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.743331    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2dg4k"]
Jul 15 13:30:33.832378 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.832349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:33.844082 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.844059    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 13:30:33.844569 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.844424    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 13:30:33.844699 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.844592    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 13:30:33.847910 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.847886    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 13:30:33.848015 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.847886    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 13:30:33.848015 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.847898    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jpzss\""
Jul 15 13:30:33.851891 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.851869    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 13:30:33.855718 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.855695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:33.941638 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.941589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:33.965545 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.965505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7fe5049-3140-4010-8905-b03a84d94d73-metrics-client-ca\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:33.965545 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.965559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7fe5049-3140-4010-8905-b03a84d94d73-root\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:33.965791 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.965586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-tls\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:33.965791 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.965682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wvn9\" (UniqueName: \"kubernetes.io/projected/a7fe5049-3140-4010-8905-b03a84d94d73-kube-api-access-4wvn9\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:33.965791 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.965738    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:33.965791 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.965780    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-textfile\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:33.965978 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.965805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-wtmp\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:33.965978 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.965864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-accelerators-collector-config\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:33.965978 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:33.965912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7fe5049-3140-4010-8905-b03a84d94d73-sys\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.066820 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.066785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.066820 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.066833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-textfile\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.067086 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.066853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-wtmp\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.067086 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.066877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-accelerators-collector-config\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.067086 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.066904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7fe5049-3140-4010-8905-b03a84d94d73-sys\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.067086 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.066949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7fe5049-3140-4010-8905-b03a84d94d73-metrics-client-ca\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.067086 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.066971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7fe5049-3140-4010-8905-b03a84d94d73-root\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.067086 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.067019    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-tls\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.067086 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.067052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wvn9\" (UniqueName: \"kubernetes.io/projected/a7fe5049-3140-4010-8905-b03a84d94d73-kube-api-access-4wvn9\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.067704 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.067673    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7fe5049-3140-4010-8905-b03a84d94d73-sys\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.067979 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.067959    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-textfile\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.068120 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.068101    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-wtmp\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.068618 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.068598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-accelerators-collector-config\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.068730 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.068663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7fe5049-3140-4010-8905-b03a84d94d73-root\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.069103 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.069083    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7fe5049-3140-4010-8905-b03a84d94d73-metrics-client-ca\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.069206 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:30:34.069171    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 15 13:30:34.069293 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:30:34.069221    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-tls podName:a7fe5049-3140-4010-8905-b03a84d94d73 nodeName:}" failed. No retries permitted until 2026-07-15 13:30:34.569207935 +0000 UTC m=+78.912142993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-tls") pod "node-exporter-2dg4k" (UID: "a7fe5049-3140-4010-8905-b03a84d94d73") : secret "node-exporter-tls" not found
Jul 15 13:30:34.071766 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.071745    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.076928 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.076892    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wvn9\" (UniqueName: \"kubernetes.io/projected/a7fe5049-3140-4010-8905-b03a84d94d73-kube-api-access-4wvn9\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.081527 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.081495    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:34.182279 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.182239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:34.296351 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.296324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:34.393596 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.393520    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:34.489868 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.489826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:34.571866 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.571833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-tls\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:34.572051 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:30:34.571962    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 15 13:30:34.572051 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:30:34.572037    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-tls podName:a7fe5049-3140-4010-8905-b03a84d94d73 nodeName:}" failed. No retries permitted until 2026-07-15 13:30:35.57201724 +0000 UTC m=+79.914952321 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-tls") pod "node-exporter-2dg4k" (UID: "a7fe5049-3140-4010-8905-b03a84d94d73") : secret "node-exporter-tls" not found
Jul 15 13:30:34.585641 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.585608    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:34.705074 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.704990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:34.955252 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:34.955142    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:35.368123 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:35.368087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:35.411595 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:35.411562    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5dd7fddbc6-bd6xz"
Jul 15 13:30:35.580179 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:35.580148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-tls\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:35.582632 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:35.582597    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7fe5049-3140-4010-8905-b03a84d94d73-node-exporter-tls\") pod \"node-exporter-2dg4k\" (UID: \"a7fe5049-3140-4010-8905-b03a84d94d73\") " pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:35.599555 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:35.599527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:35.642954 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:35.642889    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2dg4k"
Jul 15 13:30:35.853675 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:35.853628    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:35.926527 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:35.926435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:36.298293 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:36.298238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:36.396722 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:36.396592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:36.496860 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:36.496819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:36.597487 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:36.597405    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:36.700768 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:36.700727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:36.989637 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:36.989539    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:38.070126 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:38.070087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:38.249219 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:38.249182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:38.316947 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:38.316846    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:38.427708 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:38.427611    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:38.611055 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:38.611027    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-j8lb8"
Jul 15 13:30:38.675997 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:38.675930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:38.780148 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:38.780092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:39.031665 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:39.031564    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:39.096114 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:39.096076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:39.547654 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:39.547620    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:39.646791 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:39.646741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:40.049611 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:40.049572    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:40.903311 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:40.903245    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:41.008203 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.008168    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:41.221716 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:30:41.221682    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7fe5049_3140_4010_8905_b03a84d94d73.slice/crio-7c5147f56bda4d587762460dae6210b16f64a67c9bf5fb571a4a0b756f9a0174 WatchSource:0}: Error finding container 7c5147f56bda4d587762460dae6210b16f64a67c9bf5fb571a4a0b756f9a0174: Status 404 returned error can't find the container with id 7c5147f56bda4d587762460dae6210b16f64a67c9bf5fb571a4a0b756f9a0174
Jul 15 13:30:41.269638 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.269606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:41.383497 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.383469    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:41.656913 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.656878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xzbd5" event={"ID":"e918381b-1b9b-414a-80cb-a2aeac624753","Type":"ContainerStarted","Data":"e2c7b07da4efab6aec89216640d77b7fce81460c2cb8159e9dd311c0bfa42b13"}
Jul 15 13:30:41.666011 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.665416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b8ddh" event={"ID":"7ce063d2-56c7-4d5b-b506-efee6758d32c","Type":"ContainerStarted","Data":"74f85ea68aab6b9649e1d097d47b26c438c327e11d9db1abecc2220890dd10ef"}
Jul 15 13:30:41.666011 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.665496    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b8ddh" event={"ID":"7ce063d2-56c7-4d5b-b506-efee6758d32c","Type":"ContainerStarted","Data":"c5c81b647efb9339b242107ee4a18e0a6ddee656cbfbaabcb30ac6382d5bd076"}
Jul 15 13:30:41.668722 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.668695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rdb96" event={"ID":"35e809cf-4d2d-4499-829f-c9e7ef493957","Type":"ContainerStarted","Data":"e3274310e01b251487e3b131f4b57db1d47ed79b6e6cf43929e8e52cf844464a"}
Jul 15 13:30:41.671132 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.670817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-548bh" event={"ID":"1bfa268c-edc9-4417-a838-d777e9b9e940","Type":"ContainerStarted","Data":"94ab6409bb7222a35e73e5301f41568d39a10638400d013581dcede053aeda99"}
Jul 15 13:30:41.672794 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.672770    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:30:41.682345 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.682295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-4hhbg" event={"ID":"6222ec40-0d4d-460c-9528-66af0c47c175","Type":"ContainerStarted","Data":"262032fbf5c7b78b1f0a18444fb62916b34c4285b5ec1ef4c659c017bb5a1e85"}
Jul 15 13:30:41.682926 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.682905    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-4hhbg"
Jul 15 13:30:41.684540 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.684510    2577 patch_prober.go:28] interesting pod/downloads-68c9cbb497-4hhbg container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.10:8080/\": dial tcp 10.133.0.10:8080: connect: connection refused" start-of-body=
Jul 15 13:30:41.684658 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.684565    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-4hhbg" podUID="6222ec40-0d4d-460c-9528-66af0c47c175" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.10:8080/\": dial tcp 10.133.0.10:8080: connect: connection refused"
Jul 15 13:30:41.685289 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.685244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2dg4k" event={"ID":"a7fe5049-3140-4010-8905-b03a84d94d73","Type":"ContainerStarted","Data":"7c5147f56bda4d587762460dae6210b16f64a67c9bf5fb571a4a0b756f9a0174"}
Jul 15 13:30:41.685399 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.685225    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xzbd5" podStartSLOduration=70.740894177 podStartE2EDuration="1m25.685209114s" totalImagesPullingTime="14.944314937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:41.684703917 +0000 UTC m=+86.027639010" watchObservedRunningTime="2026-07-15 13:30:41.685209114 +0000 UTC m=+86.028144195"
Jul 15 13:30:41.712375 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.712321    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-548bh" podStartSLOduration=71.028866098 podStartE2EDuration="1m25.712305925s" totalImagesPullingTime="14.683439827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:41.712132906 +0000 UTC m=+86.055068005" watchObservedRunningTime="2026-07-15 13:30:41.712305925 +0000 UTC m=+86.055241005"
Jul 15 13:30:41.736224 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.736172    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b8ddh" podStartSLOduration=71.040248669 podStartE2EDuration="1m25.73615606s" totalImagesPullingTime="14.695907391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:41.735569839 +0000 UTC m=+86.078504933" watchObservedRunningTime="2026-07-15 13:30:41.73615606 +0000 UTC m=+86.079091140"
Jul 15 13:30:41.759575 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:41.759516    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-4hhbg" podStartSLOduration=1.9019213910000001 podStartE2EDuration="29.75949975s" totalImagesPullingTime="27.857578359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:41.758782782 +0000 UTC m=+86.101717857" watchObservedRunningTime="2026-07-15 13:30:41.75949975 +0000 UTC m=+86.102434842"
Jul 15 13:30:42.166122 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:42.166081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:42.261767 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:42.261680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:42.702230 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:42.702195    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-4hhbg"
Jul 15 13:30:46.712988 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:46.712946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rdb96" event={"ID":"35e809cf-4d2d-4499-829f-c9e7ef493957","Type":"ContainerStarted","Data":"660004e9f6e4145bf53154b8a7f50eee2d8d7f6761173d291911e1a95a1e2f03"}
Jul 15 13:30:46.738910 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:46.738852    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rdb96" podStartSLOduration=1.708471079 podStartE2EDuration="34.738836303s" totalImagesPullingTime="33.030365224s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:30:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:46.737235354 +0000 UTC m=+91.080170472" watchObservedRunningTime="2026-07-15 13:30:46.738836303 +0000 UTC m=+91.081771380"
Jul 15 13:30:47.175646 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:47.175609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:47.325355 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:47.325317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:47.719323 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:47.719251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2dg4k" event={"ID":"a7fe5049-3140-4010-8905-b03a84d94d73","Type":"ContainerStarted","Data":"42936d9306691c3d2d1d35e0f21503a9f3ca276db2a71c396841e912dcc523a0"}
Jul 15 13:30:48.725341 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:48.725299    2577 generic.go:356] "Generic (PLEG): container finished" podID="a7fe5049-3140-4010-8905-b03a84d94d73" containerID="42936d9306691c3d2d1d35e0f21503a9f3ca276db2a71c396841e912dcc523a0" exitCode=0
Jul 15 13:30:48.725804 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:48.725338    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2dg4k" event={"ID":"a7fe5049-3140-4010-8905-b03a84d94d73","Type":"ContainerDied","Data":"42936d9306691c3d2d1d35e0f21503a9f3ca276db2a71c396841e912dcc523a0"}
Jul 15 13:30:49.736042 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:49.735994    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2dg4k" event={"ID":"a7fe5049-3140-4010-8905-b03a84d94d73","Type":"ContainerStarted","Data":"c0c6b3a3586789c203b76fa3452c33442f18757ad364899d529ac7e15a656280"}
Jul 15 13:30:49.736533 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:49.736079    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2dg4k" event={"ID":"a7fe5049-3140-4010-8905-b03a84d94d73","Type":"ContainerStarted","Data":"33ac35017f6ae96f46f17f83daa7ae80c2dcaa852040931a888ebf1c71202474"}
Jul 15 13:30:49.804691 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:49.804656    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:49.908729 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:49.908689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:54.350203 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:54.350149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:54.436444 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:54.436402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:54.540973 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:54.540921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:55.914014 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:55.913974    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:56.013316 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:56.013281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:56.214241 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:56.214153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:30:56.319929 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:30:56.319893    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:31:03.168477 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:03.168434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:31:07.255604 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:07.255565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:31:07.352747 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:07.352710    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:31:10.913851 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:10.913815    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:31:10.983902 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:10.983869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:31:13.700033 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:13.700000    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-548bh"
Jul 15 13:31:13.722800 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:13.722729    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2dg4k" podStartSLOduration=34.296464045 podStartE2EDuration="40.722708199s" totalImagesPullingTime="6.300870741s" totalInitContainerRuntime="125.373413ms" isStatefulPod=true podCreationTimestamp="2026-07-15 13:30:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:30:49.774554196 +0000 UTC m=+94.117489277" watchObservedRunningTime="2026-07-15 13:31:13.722708199 +0000 UTC m=+118.065643280"
Jul 15 13:31:27.761088 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:27.761047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:31:49.859087 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:49.859042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:31:50.004510 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:50.004475    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:31:52.108878 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:52.108846    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:31:52.179678 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:31:52.179641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:32:00.760112 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:32:00.760076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:32:01.729736 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:32:01.729705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:32:01.853639 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:32:01.853598    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:32:02.617558 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:32:02.617514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:32:02.720216 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:32:02.720180    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:32:02.831156 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:32:02.831119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:32:10.997190 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:32:10.997102    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:32:11.200430 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:32:11.200397    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:32:44.444384 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:32:44.444345    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:00.310599 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:00.310557    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:00.406091 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:00.406051    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:38.816748 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:38.816661    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:43.844796 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:43.844758    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:44.673959 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:44.673926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:47.666991 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:47.666960    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:47.764907 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:47.764875    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:49.243863 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:49.243825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:49.388179 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:49.388138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:49.872637 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:49.872603    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:49.998906 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:49.998866    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:52.320805 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:52.320767    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:52.461698 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:52.461668    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:54.661062 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:54.661029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:54.758408 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:54.758376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:33:59.872033 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:33:59.871987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:01.922049 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:01.922016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:02.020218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:02.020188    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:07.594175 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:07.594140    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:11.353326 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:11.353288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:16.038688 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:16.038657    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:34:16.041796 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:16.041771    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:34:16.044638 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:16.044608    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 13:34:18.839530 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:18.839489    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:18.939583 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:18.939552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:19.285006 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:19.284979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:19.417878 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:19.417850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:20.748085 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:20.748047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:20.861455 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:20.861405    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:21.690547 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:21.690514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:21.784083 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:21.784047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:22.348904 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.348874    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:22.782190 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.782158    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"]
Jul 15 13:34:22.784085 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.784070    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.794372 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.794335    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 13:34:22.794713 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.794597    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cp25j\""
Jul 15 13:34:22.799475 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.799461    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 13:34:22.799936 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.799922    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 13:34:22.800013 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.799995    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 13:34:22.890094 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.890054    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"]
Jul 15 13:34:22.895029 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.894994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08f760a9-1e2a-40fe-b054-a71fbddf46a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.895177 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.895051    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qcf6\" (UniqueName: \"kubernetes.io/projected/08f760a9-1e2a-40fe-b054-a71fbddf46a5-kube-api-access-6qcf6\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.895177 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.895109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f760a9-1e2a-40fe-b054-a71fbddf46a5-tmp\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.895177 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.895141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08f760a9-1e2a-40fe-b054-a71fbddf46a5-manifests\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.895309 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.895212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08f760a9-1e2a-40fe-b054-a71fbddf46a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.934198 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.934160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:22.995594 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.995561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08f760a9-1e2a-40fe-b054-a71fbddf46a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.995695 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.995599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08f760a9-1e2a-40fe-b054-a71fbddf46a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.995695 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.995622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qcf6\" (UniqueName: \"kubernetes.io/projected/08f760a9-1e2a-40fe-b054-a71fbddf46a5-kube-api-access-6qcf6\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.995813 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.995750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f760a9-1e2a-40fe-b054-a71fbddf46a5-tmp\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.995813 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.995798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08f760a9-1e2a-40fe-b054-a71fbddf46a5-manifests\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.996195 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.996166    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08f760a9-1e2a-40fe-b054-a71fbddf46a5-manifests\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.998165 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.998139    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08f760a9-1e2a-40fe-b054-a71fbddf46a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.998417 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.998399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08f760a9-1e2a-40fe-b054-a71fbddf46a5-tmp\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:22.998646 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:22.998633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08f760a9-1e2a-40fe-b054-a71fbddf46a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:23.005073 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.005044    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qcf6\" (UniqueName: \"kubernetes.io/projected/08f760a9-1e2a-40fe-b054-a71fbddf46a5-kube-api-access-6qcf6\") pod \"opendatahub-operator-controller-manager-84978f4d9d-qm7vt\" (UID: \"08f760a9-1e2a-40fe-b054-a71fbddf46a5\") " pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:23.093791 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.093712    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:23.238055 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.238024    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"]
Jul 15 13:34:23.242138 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:34:23.242109    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f760a9_1e2a_40fe_b054_a71fbddf46a5.slice/crio-530d69f4cc2c6efeb75677fefee61c3e13a14800b28da8b6deef527fcdb79281 WatchSource:0}: Error finding container 530d69f4cc2c6efeb75677fefee61c3e13a14800b28da8b6deef527fcdb79281: Status 404 returned error can't find the container with id 530d69f4cc2c6efeb75677fefee61c3e13a14800b28da8b6deef527fcdb79281
Jul 15 13:34:23.243835 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.243816    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 13:34:23.366303 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.366192    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:23.436300 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.436247    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt" event={"ID":"08f760a9-1e2a-40fe-b054-a71fbddf46a5","Type":"ContainerStarted","Data":"530d69f4cc2c6efeb75677fefee61c3e13a14800b28da8b6deef527fcdb79281"}
Jul 15 13:34:23.480684 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.480649    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:23.584195 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.584158    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:23.684958 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.684865    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:23.832927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.832880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:23.923958 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:23.923927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:27.451995 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:27.451956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt" event={"ID":"08f760a9-1e2a-40fe-b054-a71fbddf46a5","Type":"ContainerStarted","Data":"d597df24c3fe8d8cac3259f359bf0e685972d0c5224c86ad33a43970b6980666"}
Jul 15 13:34:28.457740 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:28.457699    2577 generic.go:356] "Generic (PLEG): container finished" podID="08f760a9-1e2a-40fe-b054-a71fbddf46a5" containerID="d597df24c3fe8d8cac3259f359bf0e685972d0c5224c86ad33a43970b6980666" exitCode=0
Jul 15 13:34:28.458205 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:28.457762    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt" event={"ID":"08f760a9-1e2a-40fe-b054-a71fbddf46a5","Type":"ContainerDied","Data":"d597df24c3fe8d8cac3259f359bf0e685972d0c5224c86ad33a43970b6980666"}
Jul 15 13:34:29.464092 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:29.464054    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt" event={"ID":"08f760a9-1e2a-40fe-b054-a71fbddf46a5","Type":"ContainerStarted","Data":"d625943eae959703336322655bcff02d876254cd60e74aaf61c252441516cd73"}
Jul 15 13:34:29.464498 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:29.464146    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:29.497049 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:29.496995    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt" podStartSLOduration=2.994588447 podStartE2EDuration="7.496974208s" totalImagesPullingTime="3.46957922s" totalInitContainerRuntime="1.032806541s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:34:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:34:29.495974802 +0000 UTC m=+313.838909905" watchObservedRunningTime="2026-07-15 13:34:29.496974208 +0000 UTC m=+313.839909289"
Jul 15 13:34:30.591084 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:30.591050    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:30.691533 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:30.691501    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:40.470716 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:40.470680    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84978f4d9d-qm7vt"
Jul 15 13:34:42.119586 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.119554    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:42.255119 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.255086    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:42.753996 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.753962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:42.917320 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.917286    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"]
Jul 15 13:34:42.925579 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.925552    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:42.937959 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.937939    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 13:34:42.938061 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.937986    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 13:34:42.941378 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.941357    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 13:34:42.941525 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.941510    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kjjpt\""
Jul 15 13:34:42.942872 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.942859    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 13:34:42.983349 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:42.983319    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"]
Jul 15 13:34:43.052455 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.052367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd422153-9424-4cac-9124-c950387f80ea-tmp\") pod \"kube-auth-proxy-9ddcd897f-p6rlv\" (UID: \"cd422153-9424-4cac-9124-c950387f80ea\") " pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:43.052455 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.052415    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kkpc\" (UniqueName: \"kubernetes.io/projected/cd422153-9424-4cac-9124-c950387f80ea-kube-api-access-9kkpc\") pod \"kube-auth-proxy-9ddcd897f-p6rlv\" (UID: \"cd422153-9424-4cac-9124-c950387f80ea\") " pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:43.052455 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.052448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd422153-9424-4cac-9124-c950387f80ea-tls-certs\") pod \"kube-auth-proxy-9ddcd897f-p6rlv\" (UID: \"cd422153-9424-4cac-9124-c950387f80ea\") " pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:43.153662 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.153635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd422153-9424-4cac-9124-c950387f80ea-tmp\") pod \"kube-auth-proxy-9ddcd897f-p6rlv\" (UID: \"cd422153-9424-4cac-9124-c950387f80ea\") " pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:43.154050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.153682    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kkpc\" (UniqueName: \"kubernetes.io/projected/cd422153-9424-4cac-9124-c950387f80ea-kube-api-access-9kkpc\") pod \"kube-auth-proxy-9ddcd897f-p6rlv\" (UID: \"cd422153-9424-4cac-9124-c950387f80ea\") " pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:43.154050 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.153709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd422153-9424-4cac-9124-c950387f80ea-tls-certs\") pod \"kube-auth-proxy-9ddcd897f-p6rlv\" (UID: \"cd422153-9424-4cac-9124-c950387f80ea\") " pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:43.156119 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.156092    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd422153-9424-4cac-9124-c950387f80ea-tmp\") pod \"kube-auth-proxy-9ddcd897f-p6rlv\" (UID: \"cd422153-9424-4cac-9124-c950387f80ea\") " pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:43.156353 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.156333    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd422153-9424-4cac-9124-c950387f80ea-tls-certs\") pod \"kube-auth-proxy-9ddcd897f-p6rlv\" (UID: \"cd422153-9424-4cac-9124-c950387f80ea\") " pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:43.177117 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.177090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kkpc\" (UniqueName: \"kubernetes.io/projected/cd422153-9424-4cac-9124-c950387f80ea-kube-api-access-9kkpc\") pod \"kube-auth-proxy-9ddcd897f-p6rlv\" (UID: \"cd422153-9424-4cac-9124-c950387f80ea\") " pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:43.219628 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.219594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:43.234479 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.234444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"
Jul 15 13:34:43.270744 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.270710    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vrr7r"]
Jul 15 13:34:43.274563 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.274544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:34:43.278027 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.278004    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-bhntx\""
Jul 15 13:34:43.279059 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.279034    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 15 13:34:43.294637 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.294613    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vrr7r"]
Jul 15 13:34:43.356013 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.355883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c-cert\") pod \"odh-model-controller-7998c67dc4-vrr7r\" (UID: \"0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c\") " pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:34:43.356013 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.355931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwbjm\" (UniqueName: \"kubernetes.io/projected/0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c-kube-api-access-lwbjm\") pod \"odh-model-controller-7998c67dc4-vrr7r\" (UID: \"0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c\") " pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:34:43.361385 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.361356    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:43.387711 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.387685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv"]
Jul 15 13:34:43.391133 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:34:43.391104    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd422153_9424_4cac_9124_c950387f80ea.slice/crio-d29e2ffee22e6af1f4f1cfa1d744b703e5e06cf8024eae72ead49aed372cc59e WatchSource:0}: Error finding container d29e2ffee22e6af1f4f1cfa1d744b703e5e06cf8024eae72ead49aed372cc59e: Status 404 returned error can't find the container with id d29e2ffee22e6af1f4f1cfa1d744b703e5e06cf8024eae72ead49aed372cc59e
Jul 15 13:34:43.453803 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.453770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:43.456856 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.456837    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c-cert\") pod \"odh-model-controller-7998c67dc4-vrr7r\" (UID: \"0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c\") " pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:34:43.456942 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.456869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwbjm\" (UniqueName: \"kubernetes.io/projected/0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c-kube-api-access-lwbjm\") pod \"odh-model-controller-7998c67dc4-vrr7r\" (UID: \"0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c\") " pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:34:43.457024 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:34:43.457007    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 15 13:34:43.457097 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:34:43.457085    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c-cert podName:0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c nodeName:}" failed. No retries permitted until 2026-07-15 13:34:43.957062475 +0000 UTC m=+328.299997536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c-cert") pod "odh-model-controller-7998c67dc4-vrr7r" (UID: "0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c") : secret "odh-model-controller-webhook-cert" not found
Jul 15 13:34:43.471878 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.471850    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwbjm\" (UniqueName: \"kubernetes.io/projected/0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c-kube-api-access-lwbjm\") pod \"odh-model-controller-7998c67dc4-vrr7r\" (UID: \"0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c\") " pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:34:43.519821 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.519787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv" event={"ID":"cd422153-9424-4cac-9124-c950387f80ea","Type":"ContainerStarted","Data":"d29e2ffee22e6af1f4f1cfa1d744b703e5e06cf8024eae72ead49aed372cc59e"}
Jul 15 13:34:43.566567 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.566525    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:43.710075 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.709988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:43.960203 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.960107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c-cert\") pod \"odh-model-controller-7998c67dc4-vrr7r\" (UID: \"0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c\") " pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:34:43.962699 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:43.962669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c-cert\") pod \"odh-model-controller-7998c67dc4-vrr7r\" (UID: \"0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c\") " pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:34:44.186596 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.186553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:34:44.323482 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.323447    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vrr7r"]
Jul 15 13:34:44.327098 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:34:44.327070    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fa97da8_0c9a_4f9e_afdc_8fb4ca210d4c.slice/crio-2d3fa77885d842e0e81f71bab88b9e46dc85e468fe05d8958fe666747b1a3aa7 WatchSource:0}: Error finding container 2d3fa77885d842e0e81f71bab88b9e46dc85e468fe05d8958fe666747b1a3aa7: Status 404 returned error can't find the container with id 2d3fa77885d842e0e81f71bab88b9e46dc85e468fe05d8958fe666747b1a3aa7
Jul 15 13:34:44.378755 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.378725    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nstjz"]
Jul 15 13:34:44.383243 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.383221    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:34:44.386069 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.386050    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 15 13:34:44.402603 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.402569    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nstjz"]
Jul 15 13:34:44.403142 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.403118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:44.411121 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.411084    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"]
Jul 15 13:34:44.414420 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.414398    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.418459 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.418435    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 15 13:34:44.418578 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.418473    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 13:34:44.418964 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.418947    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 15 13:34:44.419059 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.418965    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zdfd7\""
Jul 15 13:34:44.422237 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.422217    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 15 13:34:44.446027 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.445982    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"]
Jul 15 13:34:44.465394 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.465358    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4065938b-bc79-459e-b106-815157f1dedb-tls-certs\") pod \"model-serving-api-54d89bd5fc-nstjz\" (UID: \"4065938b-bc79-459e-b106-815157f1dedb\") " pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:34:44.465575 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.465480    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdjsb\" (UniqueName: \"kubernetes.io/projected/4065938b-bc79-459e-b106-815157f1dedb-kube-api-access-pdjsb\") pod \"model-serving-api-54d89bd5fc-nstjz\" (UID: \"4065938b-bc79-459e-b106-815157f1dedb\") " pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:34:44.514809 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.514757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:44.525583 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.525542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r" event={"ID":"0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c","Type":"ContainerStarted","Data":"2d3fa77885d842e0e81f71bab88b9e46dc85e468fe05d8958fe666747b1a3aa7"}
Jul 15 13:34:44.565992 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.565960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.566167 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.566005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4065938b-bc79-459e-b106-815157f1dedb-tls-certs\") pod \"model-serving-api-54d89bd5fc-nstjz\" (UID: \"4065938b-bc79-459e-b106-815157f1dedb\") " pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:34:44.566167 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.566036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz6nd\" (UniqueName: \"kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-kube-api-access-pz6nd\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.566167 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.566077    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.566167 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.566109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdjsb\" (UniqueName: \"kubernetes.io/projected/4065938b-bc79-459e-b106-815157f1dedb-kube-api-access-pdjsb\") pod \"model-serving-api-54d89bd5fc-nstjz\" (UID: \"4065938b-bc79-459e-b106-815157f1dedb\") " pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:34:44.566167 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:34:44.566132    2577 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 15 13:34:44.566409 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:34:44.566192    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4065938b-bc79-459e-b106-815157f1dedb-tls-certs podName:4065938b-bc79-459e-b106-815157f1dedb nodeName:}" failed. No retries permitted until 2026-07-15 13:34:45.066173983 +0000 UTC m=+329.409109045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4065938b-bc79-459e-b106-815157f1dedb-tls-certs") pod "model-serving-api-54d89bd5fc-nstjz" (UID: "4065938b-bc79-459e-b106-815157f1dedb") : secret "model-serving-api-tls" not found
Jul 15 13:34:44.566409 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.566185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.566409 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.566235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-local-certs\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.566409 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.566311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-token\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.566409 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.566339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-cacerts\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.578876 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.578818    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdjsb\" (UniqueName: \"kubernetes.io/projected/4065938b-bc79-459e-b106-815157f1dedb-kube-api-access-pdjsb\") pod \"model-serving-api-54d89bd5fc-nstjz\" (UID: \"4065938b-bc79-459e-b106-815157f1dedb\") " pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:34:44.666907 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.666870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-local-certs\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.667122 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.666918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-token\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.667122 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.666939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-cacerts\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.667122 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.666969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.667122 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.667017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6nd\" (UniqueName: \"kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-kube-api-access-pz6nd\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.667122 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.667055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.667122 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.667099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.667830 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.667738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.669636 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.669617    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-local-certs\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.669747 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.669695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.669801 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.669791    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-cacerts\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.669863 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.669847    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.678593 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.678573    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-token\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.678685 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.678596    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz6nd\" (UniqueName: \"kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-kube-api-access-pz6nd\") pod \"istiod-openshift-gateway-7574459656-mwc9r\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.686917 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.686889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:44.726604 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.726569    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:44.883936 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:44.883904    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"]
Jul 15 13:34:44.886514 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:34:44.886472    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e76a4f_7c4e_4acb_9842_c3a4bacc09fa.slice/crio-dbea5c69e0e65e592e262411ebb2c9377b636e8ebfdb543b03857c55510ead6a WatchSource:0}: Error finding container dbea5c69e0e65e592e262411ebb2c9377b636e8ebfdb543b03857c55510ead6a: Status 404 returned error can't find the container with id dbea5c69e0e65e592e262411ebb2c9377b636e8ebfdb543b03857c55510ead6a
Jul 15 13:34:45.070996 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:45.070897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4065938b-bc79-459e-b106-815157f1dedb-tls-certs\") pod \"model-serving-api-54d89bd5fc-nstjz\" (UID: \"4065938b-bc79-459e-b106-815157f1dedb\") " pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:34:45.074562 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:45.074524    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4065938b-bc79-459e-b106-815157f1dedb-tls-certs\") pod \"model-serving-api-54d89bd5fc-nstjz\" (UID: \"4065938b-bc79-459e-b106-815157f1dedb\") " pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:34:45.296454 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:45.296415    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:34:45.472379 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:45.472300    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nstjz"]
Jul 15 13:34:45.532129 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:45.532083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nstjz" event={"ID":"4065938b-bc79-459e-b106-815157f1dedb","Type":"ContainerStarted","Data":"f608ea97d74e091ca01e9bb2ba956cdada7f5a204734ca1c8a2caf0b4b88b2de"}
Jul 15 13:34:45.533634 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:45.533590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r" event={"ID":"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa","Type":"ContainerStarted","Data":"dbea5c69e0e65e592e262411ebb2c9377b636e8ebfdb543b03857c55510ead6a"}
Jul 15 13:34:45.878965 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:45.878926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:49.311144 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.311104    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 15 13:34:49.311570 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.311184    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 15 13:34:49.551629 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.551532    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv" event={"ID":"cd422153-9424-4cac-9124-c950387f80ea","Type":"ContainerStarted","Data":"04a278077b51e1280f559e1c8c6f013787028fa24c917925d0f7aaa1b09671c3"}
Jul 15 13:34:49.553170 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.553141    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r" event={"ID":"0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c","Type":"ContainerStarted","Data":"fe08f744328a1583b00dd7f7c6b30f3920e04874e5c9790c5571b3f3b6966d5a"}
Jul 15 13:34:49.553324 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.553247    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:34:49.554674 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.554648    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nstjz" event={"ID":"4065938b-bc79-459e-b106-815157f1dedb","Type":"ContainerStarted","Data":"74da280d18436ba1f6e44442b4924f70437f61a0f2c6f7cf8526aae2d0c71a41"}
Jul 15 13:34:49.554785 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.554700    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:34:49.556156 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.556124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r" event={"ID":"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa","Type":"ContainerStarted","Data":"d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316"}
Jul 15 13:34:49.556286 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.556251    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:49.614027 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.613966    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9ddcd897f-p6rlv" podStartSLOduration=1.727156365 podStartE2EDuration="7.613947831s" totalImagesPullingTime="5.886791466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:34:49.60713404 +0000 UTC m=+333.950069120" watchObservedRunningTime="2026-07-15 13:34:49.613947831 +0000 UTC m=+333.956882915"
Jul 15 13:34:49.633591 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.633543    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r" podStartSLOduration=1.651614091 podStartE2EDuration="6.633525101s" totalImagesPullingTime="4.98191101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:34:49.632183255 +0000 UTC m=+333.975118361" watchObservedRunningTime="2026-07-15 13:34:49.633525101 +0000 UTC m=+333.976460258"
Jul 15 13:34:49.665026 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.664981    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r" podStartSLOduration=1.24312788 podStartE2EDuration="5.664966438s" totalImagesPullingTime="4.421838558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:34:49.664780381 +0000 UTC m=+334.007715479" watchObservedRunningTime="2026-07-15 13:34:49.664966438 +0000 UTC m=+334.007901518"
Jul 15 13:34:49.725155 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:49.725103    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-nstjz" podStartSLOduration=1.8876975360000001 podStartE2EDuration="5.725082412s" totalImagesPullingTime="3.837384876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:34:49.722586282 +0000 UTC m=+334.065521379" watchObservedRunningTime="2026-07-15 13:34:49.725082412 +0000 UTC m=+334.068017494"
Jul 15 13:34:50.562285 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:50.562232    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-mwc9r container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 13:34:50.562714 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:50.562348    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r" podUID="97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 13:34:50.582335 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:50.582298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:50.682367 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:50.682336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:50.996907 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:50.996824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:51.189920 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.189889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:51.277797 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.277756    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"]
Jul 15 13:34:51.281210 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.281185    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.298371 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.298341    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"]
Jul 15 13:34:51.418427 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.418393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:51.424819 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.424794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/026e9cca-d755-4041-bf73-03b2b2fc852d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.424927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.424823    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.424927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.424864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/026e9cca-d755-4041-bf73-03b2b2fc852d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.425007 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.424944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.425007 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.424977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.425007 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.424997    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.425094 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.425053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd8rf\" (UniqueName: \"kubernetes.io/projected/026e9cca-d755-4041-bf73-03b2b2fc852d-kube-api-access-vd8rf\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.525489 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.525452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/026e9cca-d755-4041-bf73-03b2b2fc852d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.525489 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.525494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.525723 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.525513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.525723 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.525532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.525723 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.525561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd8rf\" (UniqueName: \"kubernetes.io/projected/026e9cca-d755-4041-bf73-03b2b2fc852d-kube-api-access-vd8rf\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.525723 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.525593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/026e9cca-d755-4041-bf73-03b2b2fc852d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.525723 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.525619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.526411 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.526335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.528141 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.528091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.528218 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.528148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.528525 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.528506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/026e9cca-d755-4041-bf73-03b2b2fc852d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.528525 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.528517    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/026e9cca-d755-4041-bf73-03b2b2fc852d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.530896 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.530875    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:51.549192 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.549157    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd8rf\" (UniqueName: \"kubernetes.io/projected/026e9cca-d755-4041-bf73-03b2b2fc852d-kube-api-access-vd8rf\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.549940 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.549919    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026e9cca-d755-4041-bf73-03b2b2fc852d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p82bj\" (UID: \"026e9cca-d755-4041-bf73-03b2b2fc852d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.590209 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.590170    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:51.755740 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.755706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"]
Jul 15 13:34:51.760454 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:34:51.760425    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026e9cca_d755_4041_bf73_03b2b2fc852d.slice/crio-6107f3941ce302995646b3b31d8c1a95905f55dab9e3318a875d0447908dbd4f WatchSource:0}: Error finding container 6107f3941ce302995646b3b31d8c1a95905f55dab9e3318a875d0447908dbd4f: Status 404 returned error can't find the container with id 6107f3941ce302995646b3b31d8c1a95905f55dab9e3318a875d0447908dbd4f
Jul 15 13:34:51.763429 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.763399    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 15 13:34:51.763489 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:51.763463    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 15 13:34:52.568917 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:52.568874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj" event={"ID":"026e9cca-d755-4041-bf73-03b2b2fc852d","Type":"ContainerStarted","Data":"81f06bd920fb0298a148088fec615472efb831c99bc5f80444ca5188f596c988"}
Jul 15 13:34:52.569065 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:52.568937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj" event={"ID":"026e9cca-d755-4041-bf73-03b2b2fc852d","Type":"ContainerStarted","Data":"6107f3941ce302995646b3b31d8c1a95905f55dab9e3318a875d0447908dbd4f"}
Jul 15 13:34:52.569127 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:52.569063    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:52.570831 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:52.570811    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj"
Jul 15 13:34:52.596902 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:52.596846    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p82bj" podStartSLOduration=1.596828582 podStartE2EDuration="1.596828582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:34:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:34:52.595402715 +0000 UTC m=+336.938337825" watchObservedRunningTime="2026-07-15 13:34:52.596828582 +0000 UTC m=+336.939763662"
Jul 15 13:34:52.705958 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:52.705928    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"]
Jul 15 13:34:52.706128 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:52.706108    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r" podUID="97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" containerName="discovery" containerID="cri-o://d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316" gracePeriod=30
Jul 15 13:34:52.706944 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:52.706828    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:52.956223 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:52.956201    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:53.038664 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.038632    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-ca-configmap\") pod \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") "
Jul 15 13:34:53.038837 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.038674    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-local-certs\") pod \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") "
Jul 15 13:34:53.038837 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.038702    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-token\") pod \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") "
Jul 15 13:34:53.038837 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.038730    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-dns-cert\") pod \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") "
Jul 15 13:34:53.038837 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.038756    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-cacerts\") pod \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") "
Jul 15 13:34:53.038837 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.038781    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-kubeconfig\") pod \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") "
Jul 15 13:34:53.039052 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.038846    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz6nd\" (UniqueName: \"kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-kube-api-access-pz6nd\") pod \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\" (UID: \"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa\") "
Jul 15 13:34:53.039052 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.039035    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" (UID: "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:34:53.039176 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.039147    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-ca-configmap\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:34:53.041533 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.041404    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" (UID: "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:34:53.041533 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.041480    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-local-certs" (OuterVolumeSpecName: "local-certs") pod "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" (UID: "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:34:53.041533 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.041482    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" (UID: "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:34:53.041748 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.041570    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-cacerts" (OuterVolumeSpecName: "cacerts") pod "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" (UID: "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:34:53.041748 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.041582    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" (UID: "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:34:53.041815 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.041782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-kube-api-access-pz6nd" (OuterVolumeSpecName: "kube-api-access-pz6nd") pod "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" (UID: "97e76a4f-7c4e-4acb-9842-c3a4bacc09fa"). InnerVolumeSpecName "kube-api-access-pz6nd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:34:53.140322 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.140214    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-local-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:34:53.140322 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.140244    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-token\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:34:53.140322 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.140254    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-csr-dns-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:34:53.140322 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.140282    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-cacerts\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:34:53.140322 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.140295    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-istio-kubeconfig\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:34:53.140322 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.140308    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz6nd\" (UniqueName: \"kubernetes.io/projected/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa-kube-api-access-pz6nd\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:34:53.574125 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.574089    2577 generic.go:356] "Generic (PLEG): container finished" podID="97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" containerID="d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316" exitCode=0
Jul 15 13:34:53.574330 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.574141    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"
Jul 15 13:34:53.574330 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.574173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r" event={"ID":"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa","Type":"ContainerDied","Data":"d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316"}
Jul 15 13:34:53.574330 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.574224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r" event={"ID":"97e76a4f-7c4e-4acb-9842-c3a4bacc09fa","Type":"ContainerDied","Data":"dbea5c69e0e65e592e262411ebb2c9377b636e8ebfdb543b03857c55510ead6a"}
Jul 15 13:34:53.574330 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.574244    2577 scope.go:117] "RemoveContainer" containerID="d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316"
Jul 15 13:34:53.584560 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.584535    2577 scope.go:117] "RemoveContainer" containerID="d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316"
Jul 15 13:34:53.585025 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:34:53.585004    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316\": container with ID starting with d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316 not found: ID does not exist" containerID="d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316"
Jul 15 13:34:53.585132 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.585032    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316"} err="failed to get container status \"d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316\": rpc error: code = NotFound desc = could not find container \"d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316\": container with ID starting with d2ecc27a8b0f8208009cfd577b5486cd9032ae0cf0105bd7f7a8f22d3f4b8316 not found: ID does not exist"
Jul 15 13:34:53.601972 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.601941    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"]
Jul 15 13:34:53.606665 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:53.606643    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-mwc9r"]
Jul 15 13:34:54.145377 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.145346    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" path="/var/lib/kubelet/pods/97e76a4f-7c4e-4acb-9842-c3a4bacc09fa/volumes"
Jul 15 13:34:54.244101 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.244069    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:54.353023 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.352989    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:54.419596 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.419522    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-622x9"]
Jul 15 13:34:54.419835 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.419822    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" containerName="discovery"
Jul 15 13:34:54.419874 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.419837    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" containerName="discovery"
Jul 15 13:34:54.419903 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.419886    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="97e76a4f-7c4e-4acb-9842-c3a4bacc09fa" containerName="discovery"
Jul 15 13:34:54.423988 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.423969    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9"
Jul 15 13:34:54.430478 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.430448    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 15 13:34:54.430794 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.430781    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zbgkp\""
Jul 15 13:34:54.448843 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.448816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5kdf\" (UniqueName: \"kubernetes.io/projected/63558db3-414e-4b6f-b6a4-815cd0b91d44-kube-api-access-n5kdf\") pod \"kserve-controller-manager-d6f4b65b8-622x9\" (UID: \"63558db3-414e-4b6f-b6a4-815cd0b91d44\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9"
Jul 15 13:34:54.448950 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.448852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63558db3-414e-4b6f-b6a4-815cd0b91d44-cert\") pod \"kserve-controller-manager-d6f4b65b8-622x9\" (UID: \"63558db3-414e-4b6f-b6a4-815cd0b91d44\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9"
Jul 15 13:34:54.464827 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.464798    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-622x9"]
Jul 15 13:34:54.549573 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.549538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63558db3-414e-4b6f-b6a4-815cd0b91d44-cert\") pod \"kserve-controller-manager-d6f4b65b8-622x9\" (UID: \"63558db3-414e-4b6f-b6a4-815cd0b91d44\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9"
Jul 15 13:34:54.549781 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.549646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kdf\" (UniqueName: \"kubernetes.io/projected/63558db3-414e-4b6f-b6a4-815cd0b91d44-kube-api-access-n5kdf\") pod \"kserve-controller-manager-d6f4b65b8-622x9\" (UID: \"63558db3-414e-4b6f-b6a4-815cd0b91d44\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9"
Jul 15 13:34:54.552280 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.552232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63558db3-414e-4b6f-b6a4-815cd0b91d44-cert\") pod \"kserve-controller-manager-d6f4b65b8-622x9\" (UID: \"63558db3-414e-4b6f-b6a4-815cd0b91d44\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9"
Jul 15 13:34:54.560247 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.560216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5kdf\" (UniqueName: \"kubernetes.io/projected/63558db3-414e-4b6f-b6a4-815cd0b91d44-kube-api-access-n5kdf\") pod \"kserve-controller-manager-d6f4b65b8-622x9\" (UID: \"63558db3-414e-4b6f-b6a4-815cd0b91d44\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9"
Jul 15 13:34:54.599209 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.599042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:54.733917 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.733836    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9"
Jul 15 13:34:54.886760 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.886599    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-622x9"]
Jul 15 13:34:54.889439 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:34:54.889404    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63558db3_414e_4b6f_b6a4_815cd0b91d44.slice/crio-b57208783525642a8e7aedf0876e212adfa15affb0eba55066371c5f95b3a953 WatchSource:0}: Error finding container b57208783525642a8e7aedf0876e212adfa15affb0eba55066371c5f95b3a953: Status 404 returned error can't find the container with id b57208783525642a8e7aedf0876e212adfa15affb0eba55066371c5f95b3a953
Jul 15 13:34:54.927373 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:54.927335    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:55.589584 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:55.589544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9" event={"ID":"63558db3-414e-4b6f-b6a4-815cd0b91d44","Type":"ContainerStarted","Data":"b57208783525642a8e7aedf0876e212adfa15affb0eba55066371c5f95b3a953"}
Jul 15 13:34:55.684116 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:55.684073    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:55.776971 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:55.776936    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:55.859979 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:55.859876    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:56.211046 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:56.210966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:56.310791 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:56.310757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:34:57.600421 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:57.600329    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9" event={"ID":"63558db3-414e-4b6f-b6a4-815cd0b91d44","Type":"ContainerStarted","Data":"ea9c4347e4e6a76310f3a48516f7aaa4998ee71f78dfb9f62407251746ee65f7"}
Jul 15 13:34:57.600777 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:57.600435    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9"
Jul 15 13:34:57.646203 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:34:57.646148    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9" podStartSLOduration=1.242813239 podStartE2EDuration="3.646129726s" totalImagesPullingTime="2.403316487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:34:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:34:57.643716664 +0000 UTC m=+341.986651764" watchObservedRunningTime="2026-07-15 13:34:57.646129726 +0000 UTC m=+341.989064808"
Jul 15 13:35:00.562982 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:35:00.562946    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-vrr7r"
Jul 15 13:35:00.564966 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:35:00.564944    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-nstjz"
Jul 15 13:35:28.610240 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:35:28.610208    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-622x9"
Jul 15 13:35:43.645504 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:35:43.645470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:35:43.744979 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:35:43.744940    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:35:44.653219 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:35:44.653183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:35:44.752551 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:35:44.752516    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:35:45.670429 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:35:45.670396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:00.261201 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:00.261164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:00.323274 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:00.323231    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:11.693682 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:11.693647    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:11.794362 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:11.794330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:12.866389 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:12.866359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:25.951023 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:25.950986    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:26.507950 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:26.507911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:28.285219 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:28.285188    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:28.398596 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:28.398562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:33.164143 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:33.164108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:35.830282 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:35.830237    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr"]
Jul 15 13:36:35.832484 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:35.832461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr"
Jul 15 13:36:35.835478 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:35.835459    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 13:36:35.835801 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:35.835775    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 15 13:36:35.836682 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:35.836663    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 13:36:35.836776 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:35.836696    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-g7z9q\""
Jul 15 13:36:35.845681 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:35.845654    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr"]
Jul 15 13:36:35.911574 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:35.911538    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmhcj\" (UniqueName: \"kubernetes.io/projected/e63500dd-7460-4158-bb85-7a8c23803c95-kube-api-access-dmhcj\") pod \"dns-operator-controller-manager-864dd78c4c-t4xdr\" (UID: \"e63500dd-7460-4158-bb85-7a8c23803c95\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr"
Jul 15 13:36:36.013150 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:36.013108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmhcj\" (UniqueName: \"kubernetes.io/projected/e63500dd-7460-4158-bb85-7a8c23803c95-kube-api-access-dmhcj\") pod \"dns-operator-controller-manager-864dd78c4c-t4xdr\" (UID: \"e63500dd-7460-4158-bb85-7a8c23803c95\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr"
Jul 15 13:36:36.031632 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:36.031600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmhcj\" (UniqueName: \"kubernetes.io/projected/e63500dd-7460-4158-bb85-7a8c23803c95-kube-api-access-dmhcj\") pod \"dns-operator-controller-manager-864dd78c4c-t4xdr\" (UID: \"e63500dd-7460-4158-bb85-7a8c23803c95\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr"
Jul 15 13:36:36.141739 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:36.141650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr"
Jul 15 13:36:36.281938 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:36.281758    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr"]
Jul 15 13:36:36.284659 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:36:36.284630    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode63500dd_7460_4158_bb85_7a8c23803c95.slice/crio-11f0f901c8f579278932266e130c1c18b1a047517faa87e9ca19d9d8dcb4ccf1 WatchSource:0}: Error finding container 11f0f901c8f579278932266e130c1c18b1a047517faa87e9ca19d9d8dcb4ccf1: Status 404 returned error can't find the container with id 11f0f901c8f579278932266e130c1c18b1a047517faa87e9ca19d9d8dcb4ccf1
Jul 15 13:36:36.941067 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:36.941011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:36.966606 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:36.966571    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr" event={"ID":"e63500dd-7460-4158-bb85-7a8c23803c95","Type":"ContainerStarted","Data":"11f0f901c8f579278932266e130c1c18b1a047517faa87e9ca19d9d8dcb4ccf1"}
Jul 15 13:36:38.687275 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:38.687179    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"]
Jul 15 13:36:38.690428 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:38.690410    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:36:38.694791 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:38.694770    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-xwhqv\""
Jul 15 13:36:38.712034 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:38.712007    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"]
Jul 15 13:36:38.736318 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:38.736283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g287z\" (UniqueName: \"kubernetes.io/projected/481e1b17-5a14-4baf-86a5-95b6a217e1a8-kube-api-access-g287z\") pod \"limitador-operator-controller-manager-859d7b78bd-gfs6k\" (UID: \"481e1b17-5a14-4baf-86a5-95b6a217e1a8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:36:38.837157 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:38.837119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g287z\" (UniqueName: \"kubernetes.io/projected/481e1b17-5a14-4baf-86a5-95b6a217e1a8-kube-api-access-g287z\") pod \"limitador-operator-controller-manager-859d7b78bd-gfs6k\" (UID: \"481e1b17-5a14-4baf-86a5-95b6a217e1a8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:36:38.849156 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:38.849126    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g287z\" (UniqueName: \"kubernetes.io/projected/481e1b17-5a14-4baf-86a5-95b6a217e1a8-kube-api-access-g287z\") pod \"limitador-operator-controller-manager-859d7b78bd-gfs6k\" (UID: \"481e1b17-5a14-4baf-86a5-95b6a217e1a8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:36:38.977057 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:38.976973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr" event={"ID":"e63500dd-7460-4158-bb85-7a8c23803c95","Type":"ContainerStarted","Data":"5bff624cdffe1a44c3cb90dff23415bb592bd802938afe4510f3f568d57d8e66"}
Jul 15 13:36:38.977191 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:38.977090    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr"
Jul 15 13:36:38.996207 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:38.996159    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr" podStartSLOduration=1.908865938 podStartE2EDuration="3.996144292s" totalImagesPullingTime="2.087278354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:36:38.995427466 +0000 UTC m=+443.338362546" watchObservedRunningTime="2026-07-15 13:36:38.996144292 +0000 UTC m=+443.339079402"
Jul 15 13:36:39.000601 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:39.000583    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:36:39.108886 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:39.108858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:39.169178 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:39.169151    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"]
Jul 15 13:36:39.171242 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:36:39.171213    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod481e1b17_5a14_4baf_86a5_95b6a217e1a8.slice/crio-d0e98b5d127071414f696ca102e27fb5053e92685cedb633c617f2288b7a09f4 WatchSource:0}: Error finding container d0e98b5d127071414f696ca102e27fb5053e92685cedb633c617f2288b7a09f4: Status 404 returned error can't find the container with id d0e98b5d127071414f696ca102e27fb5053e92685cedb633c617f2288b7a09f4
Jul 15 13:36:39.441070 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:39.441026    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:39.982310 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:39.982273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k" event={"ID":"481e1b17-5a14-4baf-86a5-95b6a217e1a8","Type":"ContainerStarted","Data":"d0e98b5d127071414f696ca102e27fb5053e92685cedb633c617f2288b7a09f4"}
Jul 15 13:36:40.428610 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:40.428577    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:42.177216 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:42.177178    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:42.997901 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:42.997865    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k" event={"ID":"481e1b17-5a14-4baf-86a5-95b6a217e1a8","Type":"ContainerStarted","Data":"070fefe86d64a647a7c4a89c0077784799a776318d5271c917db2733153b3db8"}
Jul 15 13:36:42.998080 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:42.997947    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:36:43.030780 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:43.030727    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k" podStartSLOduration=1.874242115 podStartE2EDuration="5.0307097s" totalImagesPullingTime="3.156467585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:36:43.029157511 +0000 UTC m=+447.372092592" watchObservedRunningTime="2026-07-15 13:36:43.0307097 +0000 UTC m=+447.373644774"
Jul 15 13:36:44.743373 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:44.743345    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:44.844148 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:44.844113    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:44.984301 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:44.984247    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:45.095622 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:45.095587    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:48.731331 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:48.731296    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:48.832136 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:48.832097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:49.985189 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:49.985156    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t4xdr"
Jul 15 13:36:53.837084 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:53.837033    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:53.943045 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:53.943013    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:54.005111 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:54.005085    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:36:56.401149 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:56.401115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:56.502750 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:56.502714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:58.781388 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:58.781330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:36:58.883962 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:36:58.883917    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:03.230750 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.230714    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"]
Jul 15 13:37:03.234121 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.234099    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:03.237554 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.237528    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zsxrl\""
Jul 15 13:37:03.248231 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.248200    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"]
Jul 15 13:37:03.335023 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.334970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cscmg\" (UniqueName: \"kubernetes.io/projected/2d314cac-9683-4d3c-9880-591aba3e4db5-kube-api-access-cscmg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qjfdf\" (UID: \"2d314cac-9683-4d3c-9880-591aba3e4db5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:03.335023 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.335021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d314cac-9683-4d3c-9880-591aba3e4db5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qjfdf\" (UID: \"2d314cac-9683-4d3c-9880-591aba3e4db5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:03.435644 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.435597    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cscmg\" (UniqueName: \"kubernetes.io/projected/2d314cac-9683-4d3c-9880-591aba3e4db5-kube-api-access-cscmg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qjfdf\" (UID: \"2d314cac-9683-4d3c-9880-591aba3e4db5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:03.435836 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.435655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d314cac-9683-4d3c-9880-591aba3e4db5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qjfdf\" (UID: \"2d314cac-9683-4d3c-9880-591aba3e4db5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:03.436074 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.436051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d314cac-9683-4d3c-9880-591aba3e4db5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qjfdf\" (UID: \"2d314cac-9683-4d3c-9880-591aba3e4db5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:03.444941 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.444908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cscmg\" (UniqueName: \"kubernetes.io/projected/2d314cac-9683-4d3c-9880-591aba3e4db5-kube-api-access-cscmg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qjfdf\" (UID: \"2d314cac-9683-4d3c-9880-591aba3e4db5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:03.544437 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.544400    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:03.690955 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.690921    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"]
Jul 15 13:37:03.694224 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:37:03.694187    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d314cac_9683_4d3c_9880_591aba3e4db5.slice/crio-43aad01927427927c8b674806f643d8ea7e15addc88da8c632d975e559e68410 WatchSource:0}: Error finding container 43aad01927427927c8b674806f643d8ea7e15addc88da8c632d975e559e68410: Status 404 returned error can't find the container with id 43aad01927427927c8b674806f643d8ea7e15addc88da8c632d975e559e68410
Jul 15 13:37:03.902680 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.902596    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"]
Jul 15 13:37:03.909207 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.909176    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"]
Jul 15 13:37:03.926431 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.926395    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"]
Jul 15 13:37:03.931070 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.931048    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:03.954475 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.954430    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"]
Jul 15 13:37:03.955776 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.955746    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"]
Jul 15 13:37:03.956038 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.955985    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k" podUID="481e1b17-5a14-4baf-86a5-95b6a217e1a8" containerName="manager" containerID="cri-o://070fefe86d64a647a7c4a89c0077784799a776318d5271c917db2733153b3db8" gracePeriod=2
Jul 15 13:37:03.968734 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.968706    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"]
Jul 15 13:37:03.974124 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:37:03.974082    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gfs6k\" is forbidden: User \"system:node:ip-10-0-141-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-134.ec2.internal' and this object" podUID="481e1b17-5a14-4baf-86a5-95b6a217e1a8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:37:03.983500 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.983471    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh"]
Jul 15 13:37:03.983926 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.983912    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="481e1b17-5a14-4baf-86a5-95b6a217e1a8" containerName="manager"
Jul 15 13:37:03.983967 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.983931    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="481e1b17-5a14-4baf-86a5-95b6a217e1a8" containerName="manager"
Jul 15 13:37:03.984007 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.983997    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="481e1b17-5a14-4baf-86a5-95b6a217e1a8" containerName="manager"
Jul 15 13:37:03.987984 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:03.987960    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh"
Jul 15 13:37:04.005046 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.005020    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh"]
Jul 15 13:37:04.026818 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:37:04.026777    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gfs6k\" is forbidden: User \"system:node:ip-10-0-141-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-134.ec2.internal' and this object" podUID="481e1b17-5a14-4baf-86a5-95b6a217e1a8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:37:04.041382 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.041352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4871219e-528a-4a3d-846a-101e269363e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jtx5h\" (UID: \"4871219e-528a-4a3d-846a-101e269363e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:04.041534 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.041403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdjgp\" (UniqueName: \"kubernetes.io/projected/4871219e-528a-4a3d-846a-101e269363e6-kube-api-access-zdjgp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jtx5h\" (UID: \"4871219e-528a-4a3d-846a-101e269363e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:04.088483 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.088453    2577 generic.go:356] "Generic (PLEG): container finished" podID="481e1b17-5a14-4baf-86a5-95b6a217e1a8" containerID="070fefe86d64a647a7c4a89c0077784799a776318d5271c917db2733153b3db8" exitCode=0
Jul 15 13:37:04.142198 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.142163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh2w2\" (UniqueName: \"kubernetes.io/projected/be13401e-dab7-4f3c-825a-4a793deca875-kube-api-access-fh2w2\") pod \"limitador-operator-controller-manager-859d7b78bd-9xmdh\" (UID: \"be13401e-dab7-4f3c-825a-4a793deca875\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh"
Jul 15 13:37:04.142382 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.142230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4871219e-528a-4a3d-846a-101e269363e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jtx5h\" (UID: \"4871219e-528a-4a3d-846a-101e269363e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:04.142382 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.142311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdjgp\" (UniqueName: \"kubernetes.io/projected/4871219e-528a-4a3d-846a-101e269363e6-kube-api-access-zdjgp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jtx5h\" (UID: \"4871219e-528a-4a3d-846a-101e269363e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:04.142989 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.142937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4871219e-528a-4a3d-846a-101e269363e6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jtx5h\" (UID: \"4871219e-528a-4a3d-846a-101e269363e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:04.158481 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.158414    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdjgp\" (UniqueName: \"kubernetes.io/projected/4871219e-528a-4a3d-846a-101e269363e6-kube-api-access-zdjgp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-jtx5h\" (UID: \"4871219e-528a-4a3d-846a-101e269363e6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:04.229587 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.229561    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:37:04.242527 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.242494    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:04.242899 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.242874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh2w2\" (UniqueName: \"kubernetes.io/projected/be13401e-dab7-4f3c-825a-4a793deca875-kube-api-access-fh2w2\") pod \"limitador-operator-controller-manager-859d7b78bd-9xmdh\" (UID: \"be13401e-dab7-4f3c-825a-4a793deca875\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh"
Jul 15 13:37:04.253909 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.253878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh2w2\" (UniqueName: \"kubernetes.io/projected/be13401e-dab7-4f3c-825a-4a793deca875-kube-api-access-fh2w2\") pod \"limitador-operator-controller-manager-859d7b78bd-9xmdh\" (UID: \"be13401e-dab7-4f3c-825a-4a793deca875\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh"
Jul 15 13:37:04.336249 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.336217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh"
Jul 15 13:37:04.343708 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.343675    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g287z\" (UniqueName: \"kubernetes.io/projected/481e1b17-5a14-4baf-86a5-95b6a217e1a8-kube-api-access-g287z\") pod \"481e1b17-5a14-4baf-86a5-95b6a217e1a8\" (UID: \"481e1b17-5a14-4baf-86a5-95b6a217e1a8\") "
Jul 15 13:37:04.346754 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.346717    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/481e1b17-5a14-4baf-86a5-95b6a217e1a8-kube-api-access-g287z" (OuterVolumeSpecName: "kube-api-access-g287z") pod "481e1b17-5a14-4baf-86a5-95b6a217e1a8" (UID: "481e1b17-5a14-4baf-86a5-95b6a217e1a8"). InnerVolumeSpecName "kube-api-access-g287z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:37:04.444415 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.444386    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g287z\" (UniqueName: \"kubernetes.io/projected/481e1b17-5a14-4baf-86a5-95b6a217e1a8-kube-api-access-g287z\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:37:04.469420 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.469370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:04.527747 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.527701    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"]
Jul 15 13:37:04.572811 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:37:04.572545    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4871219e_528a_4a3d_846a_101e269363e6.slice/crio-e73dbf5292b09e1c58134f85b321a9386c1e1dc11fe1cb9326ca321234b4eed6 WatchSource:0}: Error finding container e73dbf5292b09e1c58134f85b321a9386c1e1dc11fe1cb9326ca321234b4eed6: Status 404 returned error can't find the container with id e73dbf5292b09e1c58134f85b321a9386c1e1dc11fe1cb9326ca321234b4eed6
Jul 15 13:37:04.725036 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:04.724963    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh"]
Jul 15 13:37:04.728159 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:37:04.728118    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe13401e_dab7_4f3c_825a_4a793deca875.slice/crio-2a6f4a93b57d17e59f1664554e15407418adfd365c094c18eba05a1ea64cb938 WatchSource:0}: Error finding container 2a6f4a93b57d17e59f1664554e15407418adfd365c094c18eba05a1ea64cb938: Status 404 returned error can't find the container with id 2a6f4a93b57d17e59f1664554e15407418adfd365c094c18eba05a1ea64cb938
Jul 15 13:37:05.003637 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:05.003586    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k" podUID="481e1b17-5a14-4baf-86a5-95b6a217e1a8" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.23:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 15 13:37:05.094415 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:05.094377    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gfs6k"
Jul 15 13:37:05.094415 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:05.094380    2577 scope.go:117] "RemoveContainer" containerID="070fefe86d64a647a7c4a89c0077784799a776318d5271c917db2733153b3db8"
Jul 15 13:37:05.095892 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:05.095861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h" event={"ID":"4871219e-528a-4a3d-846a-101e269363e6","Type":"ContainerStarted","Data":"e73dbf5292b09e1c58134f85b321a9386c1e1dc11fe1cb9326ca321234b4eed6"}
Jul 15 13:37:05.097475 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:05.097449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh" event={"ID":"be13401e-dab7-4f3c-825a-4a793deca875","Type":"ContainerStarted","Data":"d095397a2bd107942f79c7dbc329da717e77abf9728353a14adbb875c0f471ef"}
Jul 15 13:37:05.097599 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:05.097499    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh" event={"ID":"be13401e-dab7-4f3c-825a-4a793deca875","Type":"ContainerStarted","Data":"2a6f4a93b57d17e59f1664554e15407418adfd365c094c18eba05a1ea64cb938"}
Jul 15 13:37:05.097599 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:05.097577    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh"
Jul 15 13:37:05.126549 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:05.126493    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh" podStartSLOduration=2.126477684 podStartE2EDuration="2.126477684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:37:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:37:05.125830609 +0000 UTC m=+469.468765689" watchObservedRunningTime="2026-07-15 13:37:05.126477684 +0000 UTC m=+469.469412763"
Jul 15 13:37:05.215609 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:05.215577    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:05.369864 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:05.369760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:06.146496 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:06.146457    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="481e1b17-5a14-4baf-86a5-95b6a217e1a8" path="/var/lib/kubelet/pods/481e1b17-5a14-4baf-86a5-95b6a217e1a8/volumes"
Jul 15 13:37:08.846171 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:08.846141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:08.947729 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:08.947696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:09.121116 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.121076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h" event={"ID":"4871219e-528a-4a3d-846a-101e269363e6","Type":"ContainerStarted","Data":"1fde572ef50bf2a7962bdaac0a0200c6ec681d14d41a5ddcda6c53ea23c3b2dc"}
Jul 15 13:37:09.121337 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.121143    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:09.122725 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.122695    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf" podUID="2d314cac-9683-4d3c-9880-591aba3e4db5" containerName="manager" containerID="cri-o://7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5" gracePeriod=2
Jul 15 13:37:09.157403 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.157348    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h" podStartSLOduration=1.992578103 podStartE2EDuration="6.157328428s" totalImagesPullingTime="4.164750325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:37:09.155025867 +0000 UTC m=+473.497960946" watchObservedRunningTime="2026-07-15 13:37:09.157328428 +0000 UTC m=+473.500263507"
Jul 15 13:37:09.160099 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:37:09.160063    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qjfdf\" is forbidden: User \"system:node:ip-10-0-141-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-134.ec2.internal' and this object" podUID="2d314cac-9683-4d3c-9880-591aba3e4db5" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:09.375482 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.375415    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:09.378423 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:37:09.378388    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qjfdf\" is forbidden: User \"system:node:ip-10-0-141-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-134.ec2.internal' and this object" podUID="2d314cac-9683-4d3c-9880-591aba3e4db5" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:09.492385 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.492348    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cscmg\" (UniqueName: \"kubernetes.io/projected/2d314cac-9683-4d3c-9880-591aba3e4db5-kube-api-access-cscmg\") pod \"2d314cac-9683-4d3c-9880-591aba3e4db5\" (UID: \"2d314cac-9683-4d3c-9880-591aba3e4db5\") "
Jul 15 13:37:09.492556 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.492412    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d314cac-9683-4d3c-9880-591aba3e4db5-extensions-socket-volume\") pod \"2d314cac-9683-4d3c-9880-591aba3e4db5\" (UID: \"2d314cac-9683-4d3c-9880-591aba3e4db5\") "
Jul 15 13:37:09.492728 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.492693    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d314cac-9683-4d3c-9880-591aba3e4db5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2d314cac-9683-4d3c-9880-591aba3e4db5" (UID: "2d314cac-9683-4d3c-9880-591aba3e4db5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:37:09.494989 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.494943    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d314cac-9683-4d3c-9880-591aba3e4db5-kube-api-access-cscmg" (OuterVolumeSpecName: "kube-api-access-cscmg") pod "2d314cac-9683-4d3c-9880-591aba3e4db5" (UID: "2d314cac-9683-4d3c-9880-591aba3e4db5"). InnerVolumeSpecName "kube-api-access-cscmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:37:09.593976 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.593939    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cscmg\" (UniqueName: \"kubernetes.io/projected/2d314cac-9683-4d3c-9880-591aba3e4db5-kube-api-access-cscmg\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:37:09.593976 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:09.593968    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d314cac-9683-4d3c-9880-591aba3e4db5-extensions-socket-volume\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:37:10.128997 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:10.128962    2577 generic.go:356] "Generic (PLEG): container finished" podID="2d314cac-9683-4d3c-9880-591aba3e4db5" containerID="7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5" exitCode=2
Jul 15 13:37:10.129432 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:10.129005    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:10.129432 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:10.129040    2577 scope.go:117] "RemoveContainer" containerID="7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5"
Jul 15 13:37:10.131591 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:37:10.131560    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qjfdf\" is forbidden: User \"system:node:ip-10-0-141-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-134.ec2.internal' and this object" podUID="2d314cac-9683-4d3c-9880-591aba3e4db5" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:10.138471 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:10.138447    2577 scope.go:117] "RemoveContainer" containerID="7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5"
Jul 15 13:37:10.138769 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:37:10.138749    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5\": container with ID starting with 7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5 not found: ID does not exist" containerID="7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5"
Jul 15 13:37:10.138825 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:10.138779    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5"} err="failed to get container status \"7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5\": rpc error: code = NotFound desc = could not find container \"7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5\": container with ID starting with 7cdabc58a650f7c07073a8e77339f6962f6d3cb9499b8257476f49133d7180d5 not found: ID does not exist"
Jul 15 13:37:10.140371 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:37:10.140343    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qjfdf\" is forbidden: User \"system:node:ip-10-0-141-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-134.ec2.internal' and this object" podUID="2d314cac-9683-4d3c-9880-591aba3e4db5" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qjfdf"
Jul 15 13:37:10.147063 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:10.147038    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d314cac-9683-4d3c-9880-591aba3e4db5" path="/var/lib/kubelet/pods/2d314cac-9683-4d3c-9880-591aba3e4db5/volumes"
Jul 15 13:37:12.644673 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:12.644632    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:12.746547 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:12.746513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:15.467187 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:15.467144    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:15.630026 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:15.629986    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:16.108807 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:16.108776    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9xmdh"
Jul 15 13:37:20.132695 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:20.132659    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:36.632521 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:36.632475    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:36.698516 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:36.698483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:37.076301 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.076246    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"]
Jul 15 13:37:37.076544 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.076504    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h" podUID="4871219e-528a-4a3d-846a-101e269363e6" containerName="manager" containerID="cri-o://1fde572ef50bf2a7962bdaac0a0200c6ec681d14d41a5ddcda6c53ea23c3b2dc" gracePeriod=10
Jul 15 13:37:37.253066 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.253035    2577 generic.go:356] "Generic (PLEG): container finished" podID="4871219e-528a-4a3d-846a-101e269363e6" containerID="1fde572ef50bf2a7962bdaac0a0200c6ec681d14d41a5ddcda6c53ea23c3b2dc" exitCode=0
Jul 15 13:37:37.253210 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.253117    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h" event={"ID":"4871219e-528a-4a3d-846a-101e269363e6","Type":"ContainerDied","Data":"1fde572ef50bf2a7962bdaac0a0200c6ec681d14d41a5ddcda6c53ea23c3b2dc"}
Jul 15 13:37:37.330943 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.330880    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:37.423794 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.423761    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4871219e-528a-4a3d-846a-101e269363e6-extensions-socket-volume\") pod \"4871219e-528a-4a3d-846a-101e269363e6\" (UID: \"4871219e-528a-4a3d-846a-101e269363e6\") "
Jul 15 13:37:37.423794 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.423801    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdjgp\" (UniqueName: \"kubernetes.io/projected/4871219e-528a-4a3d-846a-101e269363e6-kube-api-access-zdjgp\") pod \"4871219e-528a-4a3d-846a-101e269363e6\" (UID: \"4871219e-528a-4a3d-846a-101e269363e6\") "
Jul 15 13:37:37.424230 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.424205    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4871219e-528a-4a3d-846a-101e269363e6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4871219e-528a-4a3d-846a-101e269363e6" (UID: "4871219e-528a-4a3d-846a-101e269363e6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:37:37.426095 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.426075    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4871219e-528a-4a3d-846a-101e269363e6-kube-api-access-zdjgp" (OuterVolumeSpecName: "kube-api-access-zdjgp") pod "4871219e-528a-4a3d-846a-101e269363e6" (UID: "4871219e-528a-4a3d-846a-101e269363e6"). InnerVolumeSpecName "kube-api-access-zdjgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:37:37.524822 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.524788    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4871219e-528a-4a3d-846a-101e269363e6-extensions-socket-volume\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:37:37.524822 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:37.524819    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdjgp\" (UniqueName: \"kubernetes.io/projected/4871219e-528a-4a3d-846a-101e269363e6-kube-api-access-zdjgp\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:37:38.259664 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:38.259632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h" event={"ID":"4871219e-528a-4a3d-846a-101e269363e6","Type":"ContainerDied","Data":"e73dbf5292b09e1c58134f85b321a9386c1e1dc11fe1cb9326ca321234b4eed6"}
Jul 15 13:37:38.260065 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:38.259694    2577 scope.go:117] "RemoveContainer" containerID="1fde572ef50bf2a7962bdaac0a0200c6ec681d14d41a5ddcda6c53ea23c3b2dc"
Jul 15 13:37:38.260065 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:38.259797    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"
Jul 15 13:37:38.284039 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:38.284002    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"]
Jul 15 13:37:38.288388 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:38.288356    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-jtx5h"]
Jul 15 13:37:40.146450 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:40.146407    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4871219e-528a-4a3d-846a-101e269363e6" path="/var/lib/kubelet/pods/4871219e-528a-4a3d-846a-101e269363e6/volumes"
Jul 15 13:37:53.369212 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.369176    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:53.384131 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.384096    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"]
Jul 15 13:37:53.384434 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.384421    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d314cac-9683-4d3c-9880-591aba3e4db5" containerName="manager"
Jul 15 13:37:53.384487 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.384437    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d314cac-9683-4d3c-9880-591aba3e4db5" containerName="manager"
Jul 15 13:37:53.384487 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.384455    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4871219e-528a-4a3d-846a-101e269363e6" containerName="manager"
Jul 15 13:37:53.384487 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.384463    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4871219e-528a-4a3d-846a-101e269363e6" containerName="manager"
Jul 15 13:37:53.384574 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.384516    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d314cac-9683-4d3c-9880-591aba3e4db5" containerName="manager"
Jul 15 13:37:53.384574 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.384528    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4871219e-528a-4a3d-846a-101e269363e6" containerName="manager"
Jul 15 13:37:53.387384 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.387361    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:37:53.390698 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.390675    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wd5w4\""
Jul 15 13:37:53.391040 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.391015    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 13:37:53.397355 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.397316    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"]
Jul 15 13:37:53.457358 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.457319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6bqh\" (UniqueName: \"kubernetes.io/projected/ac474317-8f73-4cab-9241-b2cc41f71551-kube-api-access-t6bqh\") pod \"limitador-limitador-64c94fd64f-lwbkt\" (UID: \"ac474317-8f73-4cab-9241-b2cc41f71551\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:37:53.457529 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.457403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac474317-8f73-4cab-9241-b2cc41f71551-config-file\") pod \"limitador-limitador-64c94fd64f-lwbkt\" (UID: \"ac474317-8f73-4cab-9241-b2cc41f71551\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:37:53.472221 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.472185    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:53.483643 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.483612    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"]
Jul 15 13:37:53.558407 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.558366    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac474317-8f73-4cab-9241-b2cc41f71551-config-file\") pod \"limitador-limitador-64c94fd64f-lwbkt\" (UID: \"ac474317-8f73-4cab-9241-b2cc41f71551\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:37:53.558581 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.558427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6bqh\" (UniqueName: \"kubernetes.io/projected/ac474317-8f73-4cab-9241-b2cc41f71551-kube-api-access-t6bqh\") pod \"limitador-limitador-64c94fd64f-lwbkt\" (UID: \"ac474317-8f73-4cab-9241-b2cc41f71551\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:37:53.559028 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.559008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac474317-8f73-4cab-9241-b2cc41f71551-config-file\") pod \"limitador-limitador-64c94fd64f-lwbkt\" (UID: \"ac474317-8f73-4cab-9241-b2cc41f71551\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:37:53.567938 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.567910    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6bqh\" (UniqueName: \"kubernetes.io/projected/ac474317-8f73-4cab-9241-b2cc41f71551-kube-api-access-t6bqh\") pod \"limitador-limitador-64c94fd64f-lwbkt\" (UID: \"ac474317-8f73-4cab-9241-b2cc41f71551\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:37:53.572673 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.572640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:53.699797 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.699713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:37:53.842594 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:53.842559    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"]
Jul 15 13:37:53.845740 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:37:53.845712    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac474317_8f73_4cab_9241_b2cc41f71551.slice/crio-9139beedbdebf7b66e34ab9e5d5733b41cced87eb15c4956abb95117ec7d9f28 WatchSource:0}: Error finding container 9139beedbdebf7b66e34ab9e5d5733b41cced87eb15c4956abb95117ec7d9f28: Status 404 returned error can't find the container with id 9139beedbdebf7b66e34ab9e5d5733b41cced87eb15c4956abb95117ec7d9f28
Jul 15 13:37:54.217989 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:54.217958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:54.300651 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:54.300618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:37:54.325412 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:54.325375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt" event={"ID":"ac474317-8f73-4cab-9241-b2cc41f71551","Type":"ContainerStarted","Data":"9139beedbdebf7b66e34ab9e5d5733b41cced87eb15c4956abb95117ec7d9f28"}
Jul 15 13:37:57.339392 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:57.339354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt" event={"ID":"ac474317-8f73-4cab-9241-b2cc41f71551","Type":"ContainerStarted","Data":"bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743"}
Jul 15 13:37:57.339895 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:57.339463    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:37:57.360684 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:37:57.360626    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt" podStartSLOduration=1.257785418 podStartE2EDuration="4.360608468s" totalImagesPullingTime="3.10282305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:37:57.358938325 +0000 UTC m=+521.701873421" watchObservedRunningTime="2026-07-15 13:37:57.360608468 +0000 UTC m=+521.703543547"
Jul 15 13:38:03.789339 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:03.789306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:08.345400 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:08.345317    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:38:10.496909 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:10.496874    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"]
Jul 15 13:38:10.497441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:10.497056    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt" podUID="ac474317-8f73-4cab-9241-b2cc41f71551" containerName="limitador" containerID="cri-o://bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743" gracePeriod=30
Jul 15 13:38:11.051841 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.051818    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:38:11.212177 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.212074    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6bqh\" (UniqueName: \"kubernetes.io/projected/ac474317-8f73-4cab-9241-b2cc41f71551-kube-api-access-t6bqh\") pod \"ac474317-8f73-4cab-9241-b2cc41f71551\" (UID: \"ac474317-8f73-4cab-9241-b2cc41f71551\") "
Jul 15 13:38:11.212177 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.212126    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac474317-8f73-4cab-9241-b2cc41f71551-config-file\") pod \"ac474317-8f73-4cab-9241-b2cc41f71551\" (UID: \"ac474317-8f73-4cab-9241-b2cc41f71551\") "
Jul 15 13:38:11.212530 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.212499    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac474317-8f73-4cab-9241-b2cc41f71551-config-file" (OuterVolumeSpecName: "config-file") pod "ac474317-8f73-4cab-9241-b2cc41f71551" (UID: "ac474317-8f73-4cab-9241-b2cc41f71551"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:38:11.214521 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.214497    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac474317-8f73-4cab-9241-b2cc41f71551-kube-api-access-t6bqh" (OuterVolumeSpecName: "kube-api-access-t6bqh") pod "ac474317-8f73-4cab-9241-b2cc41f71551" (UID: "ac474317-8f73-4cab-9241-b2cc41f71551"). InnerVolumeSpecName "kube-api-access-t6bqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:38:11.312779 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.312742    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6bqh\" (UniqueName: \"kubernetes.io/projected/ac474317-8f73-4cab-9241-b2cc41f71551-kube-api-access-t6bqh\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:38:11.312779 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.312775    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac474317-8f73-4cab-9241-b2cc41f71551-config-file\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:38:11.395934 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.395899    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac474317-8f73-4cab-9241-b2cc41f71551" containerID="bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743" exitCode=0
Jul 15 13:38:11.396095 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.395953    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"
Jul 15 13:38:11.396095 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.395959    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt" event={"ID":"ac474317-8f73-4cab-9241-b2cc41f71551","Type":"ContainerDied","Data":"bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743"}
Jul 15 13:38:11.396095 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.396091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lwbkt" event={"ID":"ac474317-8f73-4cab-9241-b2cc41f71551","Type":"ContainerDied","Data":"9139beedbdebf7b66e34ab9e5d5733b41cced87eb15c4956abb95117ec7d9f28"}
Jul 15 13:38:11.396287 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.396120    2577 scope.go:117] "RemoveContainer" containerID="bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743"
Jul 15 13:38:11.404950 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.404928    2577 scope.go:117] "RemoveContainer" containerID="bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743"
Jul 15 13:38:11.405294 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:38:11.405254    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743\": container with ID starting with bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743 not found: ID does not exist" containerID="bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743"
Jul 15 13:38:11.405378 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.405308    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743"} err="failed to get container status \"bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743\": rpc error: code = NotFound desc = could not find container \"bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743\": container with ID starting with bf6b02027b3ebdb92726baaad3d6a28f4899834fc89714bd0d8dbe0c5b593743 not found: ID does not exist"
Jul 15 13:38:11.416957 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.416925    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"]
Jul 15 13:38:11.422790 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:11.422762    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lwbkt"]
Jul 15 13:38:12.146855 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:12.146821    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac474317-8f73-4cab-9241-b2cc41f71551" path="/var/lib/kubelet/pods/ac474317-8f73-4cab-9241-b2cc41f71551/volumes"
Jul 15 13:38:13.053755 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.053721    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:13.350462 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.350378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:13.558129 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.558097    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"]
Jul 15 13:38:13.558448 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.558434    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac474317-8f73-4cab-9241-b2cc41f71551" containerName="limitador"
Jul 15 13:38:13.558448 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.558449    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac474317-8f73-4cab-9241-b2cc41f71551" containerName="limitador"
Jul 15 13:38:13.558570 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.558501    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac474317-8f73-4cab-9241-b2cc41f71551" containerName="limitador"
Jul 15 13:38:13.562998 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.562974    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"
Jul 15 13:38:13.566046 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.566023    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 13:38:13.567240 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.567219    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 13:38:13.567376 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.567238    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 15 13:38:13.567376 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.567225    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hhr4r\""
Jul 15 13:38:13.573325 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.573304    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"]
Jul 15 13:38:13.598625 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.598596    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:13.734370 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.734240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6b3d1e88-0d97-4a1f-a397-f692d06ac53e-data\") pod \"postgres-69fb4594cf-l4sdx\" (UID: \"6b3d1e88-0d97-4a1f-a397-f692d06ac53e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"
Jul 15 13:38:13.734370 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.734319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cfpf\" (UniqueName: \"kubernetes.io/projected/6b3d1e88-0d97-4a1f-a397-f692d06ac53e-kube-api-access-6cfpf\") pod \"postgres-69fb4594cf-l4sdx\" (UID: \"6b3d1e88-0d97-4a1f-a397-f692d06ac53e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"
Jul 15 13:38:13.835054 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.835013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6b3d1e88-0d97-4a1f-a397-f692d06ac53e-data\") pod \"postgres-69fb4594cf-l4sdx\" (UID: \"6b3d1e88-0d97-4a1f-a397-f692d06ac53e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"
Jul 15 13:38:13.835202 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.835057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cfpf\" (UniqueName: \"kubernetes.io/projected/6b3d1e88-0d97-4a1f-a397-f692d06ac53e-kube-api-access-6cfpf\") pod \"postgres-69fb4594cf-l4sdx\" (UID: \"6b3d1e88-0d97-4a1f-a397-f692d06ac53e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"
Jul 15 13:38:13.835522 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.835503    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6b3d1e88-0d97-4a1f-a397-f692d06ac53e-data\") pod \"postgres-69fb4594cf-l4sdx\" (UID: \"6b3d1e88-0d97-4a1f-a397-f692d06ac53e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"
Jul 15 13:38:13.855392 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.855365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cfpf\" (UniqueName: \"kubernetes.io/projected/6b3d1e88-0d97-4a1f-a397-f692d06ac53e-kube-api-access-6cfpf\") pod \"postgres-69fb4594cf-l4sdx\" (UID: \"6b3d1e88-0d97-4a1f-a397-f692d06ac53e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"
Jul 15 13:38:13.873841 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:13.873806    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"
Jul 15 13:38:14.016423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:14.016391    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"]
Jul 15 13:38:14.018652 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:38:14.018623    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3d1e88_0d97_4a1f_a397_f692d06ac53e.slice/crio-900cd5d051a81d93f5dfd06baaf2fe303db449379d6cd668cc797471c55a42e7 WatchSource:0}: Error finding container 900cd5d051a81d93f5dfd06baaf2fe303db449379d6cd668cc797471c55a42e7: Status 404 returned error can't find the container with id 900cd5d051a81d93f5dfd06baaf2fe303db449379d6cd668cc797471c55a42e7
Jul 15 13:38:14.410466 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:14.410372    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx" event={"ID":"6b3d1e88-0d97-4a1f-a397-f692d06ac53e","Type":"ContainerStarted","Data":"900cd5d051a81d93f5dfd06baaf2fe303db449379d6cd668cc797471c55a42e7"}
Jul 15 13:38:30.078900 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:30.078879    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 15 13:38:30.481541 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:30.481450    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx" event={"ID":"6b3d1e88-0d97-4a1f-a397-f692d06ac53e","Type":"ContainerStarted","Data":"a5e4062a656095ea64c09fd5e26891dd94adf66aaf89e82992fc9db1e795dd31"}
Jul 15 13:38:30.481687 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:30.481554    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"
Jul 15 13:38:30.505181 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:30.505129    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx" podStartSLOduration=1.450632272 podStartE2EDuration="17.505113749s" totalImagesPullingTime="16.054481477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:38:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:38:30.50473603 +0000 UTC m=+554.847671111" watchObservedRunningTime="2026-07-15 13:38:30.505113749 +0000 UTC m=+554.848048871"
Jul 15 13:38:36.516340 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:36.516312    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-l4sdx"
Jul 15 13:38:36.870011 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:36.869923    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:37.872202 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:37.872167    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:42.502283 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:42.502233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:45.861312 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:45.861238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:45.869351 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:45.869319    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55695cf8bd-8p2h4"]
Jul 15 13:38:45.871469 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:45.871452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:38:45.877324 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:45.877296    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bphdb\""
Jul 15 13:38:45.879042 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:45.879018    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 13:38:45.879042 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:45.879033    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 13:38:45.889497 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:45.889459    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55695cf8bd-8p2h4"]
Jul 15 13:38:45.985321 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:45.985258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p46tb\" (UniqueName: \"kubernetes.io/projected/5380b47e-48c1-43a8-bc71-9a6ba57977bc-kube-api-access-p46tb\") pod \"maas-controller-55695cf8bd-8p2h4\" (UID: \"5380b47e-48c1-43a8-bc71-9a6ba57977bc\") " pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:38:45.985321 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:45.985318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5380b47e-48c1-43a8-bc71-9a6ba57977bc-webhook-cert\") pod \"maas-controller-55695cf8bd-8p2h4\" (UID: \"5380b47e-48c1-43a8-bc71-9a6ba57977bc\") " pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:38:45.987482 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:45.987454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:46.085198 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.085160    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:46.086296 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.086278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p46tb\" (UniqueName: \"kubernetes.io/projected/5380b47e-48c1-43a8-bc71-9a6ba57977bc-kube-api-access-p46tb\") pod \"maas-controller-55695cf8bd-8p2h4\" (UID: \"5380b47e-48c1-43a8-bc71-9a6ba57977bc\") " pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:38:46.086385 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.086307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5380b47e-48c1-43a8-bc71-9a6ba57977bc-webhook-cert\") pod \"maas-controller-55695cf8bd-8p2h4\" (UID: \"5380b47e-48c1-43a8-bc71-9a6ba57977bc\") " pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:38:46.086385 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.086314    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-577b8d5d55-ktvhg"]
Jul 15 13:38:46.088495 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.088472    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:38:46.089015 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.088998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5380b47e-48c1-43a8-bc71-9a6ba57977bc-webhook-cert\") pod \"maas-controller-55695cf8bd-8p2h4\" (UID: \"5380b47e-48c1-43a8-bc71-9a6ba57977bc\") " pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:38:46.100743 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.100712    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p46tb\" (UniqueName: \"kubernetes.io/projected/5380b47e-48c1-43a8-bc71-9a6ba57977bc-kube-api-access-p46tb\") pod \"maas-controller-55695cf8bd-8p2h4\" (UID: \"5380b47e-48c1-43a8-bc71-9a6ba57977bc\") " pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:38:46.106359 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.106327    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577b8d5d55-ktvhg"]
Jul 15 13:38:46.182399 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.182306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:38:46.187556 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.187534    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bc7608d-ada9-4c0b-a36e-e501a76136df-webhook-cert\") pod \"maas-controller-577b8d5d55-ktvhg\" (UID: \"3bc7608d-ada9-4c0b-a36e-e501a76136df\") " pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:38:46.187640 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.187572    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6htfz\" (UniqueName: \"kubernetes.io/projected/3bc7608d-ada9-4c0b-a36e-e501a76136df-kube-api-access-6htfz\") pod \"maas-controller-577b8d5d55-ktvhg\" (UID: \"3bc7608d-ada9-4c0b-a36e-e501a76136df\") " pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:38:46.288790 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.288749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bc7608d-ada9-4c0b-a36e-e501a76136df-webhook-cert\") pod \"maas-controller-577b8d5d55-ktvhg\" (UID: \"3bc7608d-ada9-4c0b-a36e-e501a76136df\") " pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:38:46.288949 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.288824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6htfz\" (UniqueName: \"kubernetes.io/projected/3bc7608d-ada9-4c0b-a36e-e501a76136df-kube-api-access-6htfz\") pod \"maas-controller-577b8d5d55-ktvhg\" (UID: \"3bc7608d-ada9-4c0b-a36e-e501a76136df\") " pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:38:46.292092 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.292061    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bc7608d-ada9-4c0b-a36e-e501a76136df-webhook-cert\") pod \"maas-controller-577b8d5d55-ktvhg\" (UID: \"3bc7608d-ada9-4c0b-a36e-e501a76136df\") " pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:38:46.299561 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.299528    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6htfz\" (UniqueName: \"kubernetes.io/projected/3bc7608d-ada9-4c0b-a36e-e501a76136df-kube-api-access-6htfz\") pod \"maas-controller-577b8d5d55-ktvhg\" (UID: \"3bc7608d-ada9-4c0b-a36e-e501a76136df\") " pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:38:46.324244 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.324216    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55695cf8bd-8p2h4"]
Jul 15 13:38:46.326649 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:38:46.326612    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5380b47e_48c1_43a8_bc71_9a6ba57977bc.slice/crio-961f7ad45f2d9cfe591bbe84f23a01fdc8206bae551c2247c04e667cdbb72e04 WatchSource:0}: Error finding container 961f7ad45f2d9cfe591bbe84f23a01fdc8206bae551c2247c04e667cdbb72e04: Status 404 returned error can't find the container with id 961f7ad45f2d9cfe591bbe84f23a01fdc8206bae551c2247c04e667cdbb72e04
Jul 15 13:38:46.413525 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.413489    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:38:46.551911 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.551878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55695cf8bd-8p2h4" event={"ID":"5380b47e-48c1-43a8-bc71-9a6ba57977bc","Type":"ContainerStarted","Data":"961f7ad45f2d9cfe591bbe84f23a01fdc8206bae551c2247c04e667cdbb72e04"}
Jul 15 13:38:46.560226 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:46.560202    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577b8d5d55-ktvhg"]
Jul 15 13:38:46.561583 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:38:46.561557    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc7608d_ada9_4c0b_a36e_e501a76136df.slice/crio-3f7fd0c3483c5cc19fc63959f66322e75619b564e644e5aac5678de9e624e439 WatchSource:0}: Error finding container 3f7fd0c3483c5cc19fc63959f66322e75619b564e644e5aac5678de9e624e439: Status 404 returned error can't find the container with id 3f7fd0c3483c5cc19fc63959f66322e75619b564e644e5aac5678de9e624e439
Jul 15 13:38:47.559642 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:47.559605    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577b8d5d55-ktvhg" event={"ID":"3bc7608d-ada9-4c0b-a36e-e501a76136df","Type":"ContainerStarted","Data":"3f7fd0c3483c5cc19fc63959f66322e75619b564e644e5aac5678de9e624e439"}
Jul 15 13:38:49.494421 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:49.494328    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:49.570943 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:49.570906    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55695cf8bd-8p2h4" event={"ID":"5380b47e-48c1-43a8-bc71-9a6ba57977bc","Type":"ContainerStarted","Data":"610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd"}
Jul 15 13:38:49.571161 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:49.570974    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:38:49.572353 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:49.572327    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577b8d5d55-ktvhg" event={"ID":"3bc7608d-ada9-4c0b-a36e-e501a76136df","Type":"ContainerStarted","Data":"fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc"}
Jul 15 13:38:49.572448 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:49.572422    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:38:49.605475 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:49.605425    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:38:49.630745 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:49.630698    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-577b8d5d55-ktvhg" podStartSLOduration=0.95290004 podStartE2EDuration="3.630681249s" totalImagesPullingTime="2.677781209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:38:49.630624063 +0000 UTC m=+573.973559145" watchObservedRunningTime="2026-07-15 13:38:49.630681249 +0000 UTC m=+573.973616326"
Jul 15 13:38:49.631208 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:38:49.631182    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55695cf8bd-8p2h4" podStartSLOduration=1.7294337720000001 podStartE2EDuration="4.631172035s" totalImagesPullingTime="2.901738263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:38:49.604838884 +0000 UTC m=+573.947773966" watchObservedRunningTime="2026-07-15 13:38:49.631172035 +0000 UTC m=+573.974107114"
Jul 15 13:39:00.578103 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:00.578073    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:39:00.578582 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:00.578141    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:39:00.710235 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:00.708680    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55695cf8bd-8p2h4"]
Jul 15 13:39:00.710235 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:00.708999    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55695cf8bd-8p2h4" podUID="5380b47e-48c1-43a8-bc71-9a6ba57977bc" containerName="manager" containerID="cri-o://610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd" gracePeriod=30
Jul 15 13:39:00.954599 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:00.954569    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:39:01.007810 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.007777    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p46tb\" (UniqueName: \"kubernetes.io/projected/5380b47e-48c1-43a8-bc71-9a6ba57977bc-kube-api-access-p46tb\") pod \"5380b47e-48c1-43a8-bc71-9a6ba57977bc\" (UID: \"5380b47e-48c1-43a8-bc71-9a6ba57977bc\") "
Jul 15 13:39:01.007990 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.007836    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5380b47e-48c1-43a8-bc71-9a6ba57977bc-webhook-cert\") pod \"5380b47e-48c1-43a8-bc71-9a6ba57977bc\" (UID: \"5380b47e-48c1-43a8-bc71-9a6ba57977bc\") "
Jul 15 13:39:01.010228 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.010201    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5380b47e-48c1-43a8-bc71-9a6ba57977bc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5380b47e-48c1-43a8-bc71-9a6ba57977bc" (UID: "5380b47e-48c1-43a8-bc71-9a6ba57977bc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:39:01.010228 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.010216    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5380b47e-48c1-43a8-bc71-9a6ba57977bc-kube-api-access-p46tb" (OuterVolumeSpecName: "kube-api-access-p46tb") pod "5380b47e-48c1-43a8-bc71-9a6ba57977bc" (UID: "5380b47e-48c1-43a8-bc71-9a6ba57977bc"). InnerVolumeSpecName "kube-api-access-p46tb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:39:01.109287 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.109218    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5380b47e-48c1-43a8-bc71-9a6ba57977bc-webhook-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:39:01.109287 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.109290    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p46tb\" (UniqueName: \"kubernetes.io/projected/5380b47e-48c1-43a8-bc71-9a6ba57977bc-kube-api-access-p46tb\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:39:01.624206 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.624164    2577 generic.go:356] "Generic (PLEG): container finished" podID="5380b47e-48c1-43a8-bc71-9a6ba57977bc" containerID="610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd" exitCode=0
Jul 15 13:39:01.624721 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.624211    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55695cf8bd-8p2h4"
Jul 15 13:39:01.624721 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.624251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55695cf8bd-8p2h4" event={"ID":"5380b47e-48c1-43a8-bc71-9a6ba57977bc","Type":"ContainerDied","Data":"610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd"}
Jul 15 13:39:01.624721 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.624348    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55695cf8bd-8p2h4" event={"ID":"5380b47e-48c1-43a8-bc71-9a6ba57977bc","Type":"ContainerDied","Data":"961f7ad45f2d9cfe591bbe84f23a01fdc8206bae551c2247c04e667cdbb72e04"}
Jul 15 13:39:01.624721 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.624386    2577 scope.go:117] "RemoveContainer" containerID="610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd"
Jul 15 13:39:01.632778 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.632756    2577 scope.go:117] "RemoveContainer" containerID="610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd"
Jul 15 13:39:01.633064 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:39:01.633048    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd\": container with ID starting with 610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd not found: ID does not exist" containerID="610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd"
Jul 15 13:39:01.633099 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.633074    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd"} err="failed to get container status \"610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd\": rpc error: code = NotFound desc = could not find container \"610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd\": container with ID starting with 610fbfe349bc8410594c20898d3eb8cc484bc7e7b6d311df69981e8a105d68dd not found: ID does not exist"
Jul 15 13:39:01.648427 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.648398    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55695cf8bd-8p2h4"]
Jul 15 13:39:01.652668 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:01.652644    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55695cf8bd-8p2h4"]
Jul 15 13:39:02.146592 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:02.146551    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5380b47e-48c1-43a8-bc71-9a6ba57977bc" path="/var/lib/kubelet/pods/5380b47e-48c1-43a8-bc71-9a6ba57977bc/volumes"
Jul 15 13:39:03.630657 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:03.630622    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:16.063884 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:16.063855    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:39:16.069418 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:16.069390    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:39:18.813891 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:18.813855    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:20.866534 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:20.866496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:21.359926 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:21.359892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:21.465001 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:21.464963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:21.943593 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:21.943565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:22.060323 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.060255    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:22.063230 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.063195    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"]
Jul 15 13:39:22.063540 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.063527    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5380b47e-48c1-43a8-bc71-9a6ba57977bc" containerName="manager"
Jul 15 13:39:22.063586 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.063542    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5380b47e-48c1-43a8-bc71-9a6ba57977bc" containerName="manager"
Jul 15 13:39:22.063624 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.063607    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5380b47e-48c1-43a8-bc71-9a6ba57977bc" containerName="manager"
Jul 15 13:39:22.065929 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.065913    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.069999 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.069972    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tqpmm\""
Jul 15 13:39:22.069999 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.069974    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 15 13:39:22.091731 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.091702    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"]
Jul 15 13:39:22.188213 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.188173    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e5cf3e-aeb7-4bd1-8406-1b721a84a206-maas-api-tls\") pod \"maas-api-868cd55b6f-h6c4z\" (UID: \"51e5cf3e-aeb7-4bd1-8406-1b721a84a206\") " pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.188406 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.188225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltvvf\" (UniqueName: \"kubernetes.io/projected/51e5cf3e-aeb7-4bd1-8406-1b721a84a206-kube-api-access-ltvvf\") pod \"maas-api-868cd55b6f-h6c4z\" (UID: \"51e5cf3e-aeb7-4bd1-8406-1b721a84a206\") " pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.188406 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.188373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e5cf3e-aeb7-4bd1-8406-1b721a84a206-openshift-service-ca\") pod \"maas-api-868cd55b6f-h6c4z\" (UID: \"51e5cf3e-aeb7-4bd1-8406-1b721a84a206\") " pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.289630 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.289592    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e5cf3e-aeb7-4bd1-8406-1b721a84a206-openshift-service-ca\") pod \"maas-api-868cd55b6f-h6c4z\" (UID: \"51e5cf3e-aeb7-4bd1-8406-1b721a84a206\") " pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.289815 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.289647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e5cf3e-aeb7-4bd1-8406-1b721a84a206-maas-api-tls\") pod \"maas-api-868cd55b6f-h6c4z\" (UID: \"51e5cf3e-aeb7-4bd1-8406-1b721a84a206\") " pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.289864 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.289819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltvvf\" (UniqueName: \"kubernetes.io/projected/51e5cf3e-aeb7-4bd1-8406-1b721a84a206-kube-api-access-ltvvf\") pod \"maas-api-868cd55b6f-h6c4z\" (UID: \"51e5cf3e-aeb7-4bd1-8406-1b721a84a206\") " pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.290317 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.290297    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e5cf3e-aeb7-4bd1-8406-1b721a84a206-openshift-service-ca\") pod \"maas-api-868cd55b6f-h6c4z\" (UID: \"51e5cf3e-aeb7-4bd1-8406-1b721a84a206\") " pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.292365 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.292347    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e5cf3e-aeb7-4bd1-8406-1b721a84a206-maas-api-tls\") pod \"maas-api-868cd55b6f-h6c4z\" (UID: \"51e5cf3e-aeb7-4bd1-8406-1b721a84a206\") " pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.299839 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.299815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltvvf\" (UniqueName: \"kubernetes.io/projected/51e5cf3e-aeb7-4bd1-8406-1b721a84a206-kube-api-access-ltvvf\") pod \"maas-api-868cd55b6f-h6c4z\" (UID: \"51e5cf3e-aeb7-4bd1-8406-1b721a84a206\") " pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.376440 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.376403    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:22.485551 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.485517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:22.526196 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.526172    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"]
Jul 15 13:39:22.528976 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:39:22.528951    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e5cf3e_aeb7_4bd1_8406_1b721a84a206.slice/crio-168aa691764754e87887df19fc0532f555df1d49531b98601fc9d8405c513907 WatchSource:0}: Error finding container 168aa691764754e87887df19fc0532f555df1d49531b98601fc9d8405c513907: Status 404 returned error can't find the container with id 168aa691764754e87887df19fc0532f555df1d49531b98601fc9d8405c513907
Jul 15 13:39:22.582492 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.582409    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:22.632714 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.632647    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-nclmh"]
Jul 15 13:39:22.637088 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.637067    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh"
Jul 15 13:39:22.642207 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.642182    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 15 13:39:22.642374 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.642316    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nww5q\""
Jul 15 13:39:22.646017 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.645993    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-nclmh"]
Jul 15 13:39:22.679886 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.679850    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"]
Jul 15 13:39:22.682845 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.682828    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"
Jul 15 13:39:22.694930 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.694897    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"]
Jul 15 13:39:22.714112 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.714073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z" event={"ID":"51e5cf3e-aeb7-4bd1-8406-1b721a84a206","Type":"ContainerStarted","Data":"168aa691764754e87887df19fc0532f555df1d49531b98601fc9d8405c513907"}
Jul 15 13:39:22.794241 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.794199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4303d8a9-263a-47c5-ae3a-01fba4ac6e42-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-nclmh\" (UID: \"4303d8a9-263a-47c5-ae3a-01fba4ac6e42\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh"
Jul 15 13:39:22.794441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.794292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac19ece-4c41-4a36-b9f3-c19f1235888c-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-5r5gk\" (UID: \"8ac19ece-4c41-4a36-b9f3-c19f1235888c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"
Jul 15 13:39:22.794441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.794343    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6l2q\" (UniqueName: \"kubernetes.io/projected/8ac19ece-4c41-4a36-b9f3-c19f1235888c-kube-api-access-s6l2q\") pod \"payload-pre-processing-74584bb9f8-5r5gk\" (UID: \"8ac19ece-4c41-4a36-b9f3-c19f1235888c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"
Jul 15 13:39:22.794441 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.794380    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nj55\" (UniqueName: \"kubernetes.io/projected/4303d8a9-263a-47c5-ae3a-01fba4ac6e42-kube-api-access-4nj55\") pod \"payload-processing-f9b8d9b9b-nclmh\" (UID: \"4303d8a9-263a-47c5-ae3a-01fba4ac6e42\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh"
Jul 15 13:39:22.895685 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.895579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac19ece-4c41-4a36-b9f3-c19f1235888c-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-5r5gk\" (UID: \"8ac19ece-4c41-4a36-b9f3-c19f1235888c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"
Jul 15 13:39:22.895685 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.895632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6l2q\" (UniqueName: \"kubernetes.io/projected/8ac19ece-4c41-4a36-b9f3-c19f1235888c-kube-api-access-s6l2q\") pod \"payload-pre-processing-74584bb9f8-5r5gk\" (UID: \"8ac19ece-4c41-4a36-b9f3-c19f1235888c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"
Jul 15 13:39:22.895910 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.895759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nj55\" (UniqueName: \"kubernetes.io/projected/4303d8a9-263a-47c5-ae3a-01fba4ac6e42-kube-api-access-4nj55\") pod \"payload-processing-f9b8d9b9b-nclmh\" (UID: \"4303d8a9-263a-47c5-ae3a-01fba4ac6e42\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh"
Jul 15 13:39:22.895910 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.895817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4303d8a9-263a-47c5-ae3a-01fba4ac6e42-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-nclmh\" (UID: \"4303d8a9-263a-47c5-ae3a-01fba4ac6e42\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh"
Jul 15 13:39:22.896537 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.896518    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac19ece-4c41-4a36-b9f3-c19f1235888c-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-5r5gk\" (UID: \"8ac19ece-4c41-4a36-b9f3-c19f1235888c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"
Jul 15 13:39:22.896587 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.896541    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4303d8a9-263a-47c5-ae3a-01fba4ac6e42-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-nclmh\" (UID: \"4303d8a9-263a-47c5-ae3a-01fba4ac6e42\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh"
Jul 15 13:39:22.905768 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.905741    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nj55\" (UniqueName: \"kubernetes.io/projected/4303d8a9-263a-47c5-ae3a-01fba4ac6e42-kube-api-access-4nj55\") pod \"payload-processing-f9b8d9b9b-nclmh\" (UID: \"4303d8a9-263a-47c5-ae3a-01fba4ac6e42\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh"
Jul 15 13:39:22.905896 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.905741    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6l2q\" (UniqueName: \"kubernetes.io/projected/8ac19ece-4c41-4a36-b9f3-c19f1235888c-kube-api-access-s6l2q\") pod \"payload-pre-processing-74584bb9f8-5r5gk\" (UID: \"8ac19ece-4c41-4a36-b9f3-c19f1235888c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"
Jul 15 13:39:22.948595 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.948560    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh"
Jul 15 13:39:22.993058 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:22.993022    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"
Jul 15 13:39:23.101315 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:23.101255    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-nclmh"]
Jul 15 13:39:23.105177 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:39:23.105149    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4303d8a9_263a_47c5_ae3a_01fba4ac6e42.slice/crio-290651ee72354b2e0d0fe13becb5b5b27e1cb71078912bb8d28d8010052ae2ee WatchSource:0}: Error finding container 290651ee72354b2e0d0fe13becb5b5b27e1cb71078912bb8d28d8010052ae2ee: Status 404 returned error can't find the container with id 290651ee72354b2e0d0fe13becb5b5b27e1cb71078912bb8d28d8010052ae2ee
Jul 15 13:39:23.169012 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:23.168935    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"]
Jul 15 13:39:23.172326 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:39:23.172297    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ac19ece_4c41_4a36_b9f3_c19f1235888c.slice/crio-9342f8db012592d8da450701d09553a4c20e240b3c5fe30796ba08b2408a9f96 WatchSource:0}: Error finding container 9342f8db012592d8da450701d09553a4c20e240b3c5fe30796ba08b2408a9f96: Status 404 returned error can't find the container with id 9342f8db012592d8da450701d09553a4c20e240b3c5fe30796ba08b2408a9f96
Jul 15 13:39:23.721331 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:23.721291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh" event={"ID":"4303d8a9-263a-47c5-ae3a-01fba4ac6e42","Type":"ContainerStarted","Data":"290651ee72354b2e0d0fe13becb5b5b27e1cb71078912bb8d28d8010052ae2ee"}
Jul 15 13:39:23.722454 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:23.722430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk" event={"ID":"8ac19ece-4c41-4a36-b9f3-c19f1235888c","Type":"ContainerStarted","Data":"9342f8db012592d8da450701d09553a4c20e240b3c5fe30796ba08b2408a9f96"}
Jul 15 13:39:24.729316 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:24.729256    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh" event={"ID":"4303d8a9-263a-47c5-ae3a-01fba4ac6e42","Type":"ContainerStarted","Data":"3218c3f579ce53417051c922b93de7d3d3912ba63237cc8fea666144061a2542"}
Jul 15 13:39:24.729733 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:24.729355    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh"
Jul 15 13:39:24.730833 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:24.730811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk" event={"ID":"8ac19ece-4c41-4a36-b9f3-c19f1235888c","Type":"ContainerStarted","Data":"259d44b7087bb11371d57334cce6eea05cd98b7179c7bebfa87c0490e96b78da"}
Jul 15 13:39:24.730872 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:24.730864    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"
Jul 15 13:39:24.765435 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:24.765383    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh" podStartSLOduration=1.380255406 podStartE2EDuration="2.765364986s" totalImagesPullingTime="1.38510958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:39:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:39:24.74711648 +0000 UTC m=+609.090051581" watchObservedRunningTime="2026-07-15 13:39:24.765364986 +0000 UTC m=+609.108300066"
Jul 15 13:39:24.765868 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:24.765828    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk" podStartSLOduration=1.444634468 podStartE2EDuration="2.765817168s" totalImagesPullingTime="1.3211827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:39:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:39:24.764424615 +0000 UTC m=+609.107359751" watchObservedRunningTime="2026-07-15 13:39:24.765817168 +0000 UTC m=+609.108752248"
Jul 15 13:39:26.740392 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:26.740355    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z" event={"ID":"51e5cf3e-aeb7-4bd1-8406-1b721a84a206","Type":"ContainerStarted","Data":"b38d97127f1b9702c2184fa6b266a9e2bb3a6eb15e6740ae7c99ad6149a8677d"}
Jul 15 13:39:26.740776 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:26.740456    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:26.766681 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:26.766629    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z" podStartSLOduration=1.550416255 podStartE2EDuration="4.766611668s" totalImagesPullingTime="3.216195413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:39:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:39:26.765746372 +0000 UTC m=+611.108681451" watchObservedRunningTime="2026-07-15 13:39:26.766611668 +0000 UTC m=+611.109546747"
Jul 15 13:39:29.938040 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:29.938004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:30.019162 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:30.019129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:32.750898 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:32.750869    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-868cd55b6f-h6c4z"
Jul 15 13:39:34.475899 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:34.475862    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:34.569434 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:34.569395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:35.736885 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:35.736858    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-nclmh"
Jul 15 13:39:35.737388 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:35.737080    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-5r5gk"
Jul 15 13:39:39.487333 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:39.487229    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:47.691219 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.691172    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"]
Jul 15 13:39:47.694554 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.694538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.698787 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.698768    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 13:39:47.698787 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.698775    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 15 13:39:47.699011 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.698775    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bdn7z\""
Jul 15 13:39:47.699011 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.698876    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 13:39:47.711526 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.711499    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"]
Jul 15 13:39:47.738059 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.738008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:47.799119 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.799075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f996e924-7444-4b53-aa47-ed17a3b3a8e8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.799361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.799132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.799361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.799174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.799361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.799193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.799361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.799223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjzrk\" (UniqueName: \"kubernetes.io/projected/f996e924-7444-4b53-aa47-ed17a3b3a8e8-kube-api-access-xjzrk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.799361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.799343    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.900618 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.900564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f996e924-7444-4b53-aa47-ed17a3b3a8e8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.900618 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.900629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.900882 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.900670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.900882 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.900696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.900882 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.900733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjzrk\" (UniqueName: \"kubernetes.io/projected/f996e924-7444-4b53-aa47-ed17a3b3a8e8-kube-api-access-xjzrk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.900882 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.900769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.901139 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.901113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.901139 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.901136    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.901535 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.901182    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.903130 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.903106    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f996e924-7444-4b53-aa47-ed17a3b3a8e8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.903383 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.903366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f996e924-7444-4b53-aa47-ed17a3b3a8e8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:47.918843 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:47.918807    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjzrk\" (UniqueName: \"kubernetes.io/projected/f996e924-7444-4b53-aa47-ed17a3b3a8e8-kube-api-access-xjzrk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vsxqh\" (UID: \"f996e924-7444-4b53-aa47-ed17a3b3a8e8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:48.005427 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:48.005386    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:48.159483 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:48.159459    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"]
Jul 15 13:39:48.161582 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:39:48.161553    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf996e924_7444_4b53_aa47_ed17a3b3a8e8.slice/crio-4f4eef9f1091c1cf0b637da74e2f3c4cedfc6737bcf0bd57aa60e78dcd8de970 WatchSource:0}: Error finding container 4f4eef9f1091c1cf0b637da74e2f3c4cedfc6737bcf0bd57aa60e78dcd8de970: Status 404 returned error can't find the container with id 4f4eef9f1091c1cf0b637da74e2f3c4cedfc6737bcf0bd57aa60e78dcd8de970
Jul 15 13:39:48.162868 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:48.162850    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 13:39:48.843034 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:48.840795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh" event={"ID":"f996e924-7444-4b53-aa47-ed17a3b3a8e8","Type":"ContainerStarted","Data":"4f4eef9f1091c1cf0b637da74e2f3c4cedfc6737bcf0bd57aa60e78dcd8de970"}
Jul 15 13:39:49.850627 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:49.850595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh" event={"ID":"f996e924-7444-4b53-aa47-ed17a3b3a8e8","Type":"ContainerStarted","Data":"770e585c129202ef0e7ec3c08808518809cb73c1f00d136f463a5d2080afdea7"}
Jul 15 13:39:52.992094 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:52.992060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:39:58.005788 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:58.005748    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:58.006214 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:58.005851    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:58.018470 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:58.018435    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:39:58.041688 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:58.041630    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh" podStartSLOduration=9.869481627 podStartE2EDuration="11.041609804s" totalImagesPullingTime="1.172128177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:39:49.879064226 +0000 UTC m=+634.221999306" watchObservedRunningTime="2026-07-15 13:39:58.041609804 +0000 UTC m=+642.384544890"
Jul 15 13:39:58.902060 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:39:58.902023    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vsxqh"
Jul 15 13:40:04.585932 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:04.585881    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:40:19.079646 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.079615    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"]
Jul 15 13:40:19.083322 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.083301    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.086657 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.086623    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 15 13:40:19.097291 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.097214    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"]
Jul 15 13:40:19.111866 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.111831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:40:19.176034 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.176000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zxxd\" (UniqueName: \"kubernetes.io/projected/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-kube-api-access-9zxxd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.176034 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.176037    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.176247 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.176058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.176247 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.176153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.176247 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.176215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.176378 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.176256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.276876 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.276834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.277073 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.276891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.277073 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.276917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zxxd\" (UniqueName: \"kubernetes.io/projected/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-kube-api-access-9zxxd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.277073 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.276936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.277073 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.276954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.277073 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.276984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.277384 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.277366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.277446 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.277423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.277488 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.277445    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.279329 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.279311    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.279594 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.279577    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.287158 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.287133    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zxxd\" (UniqueName: \"kubernetes.io/projected/7c7df76b-e600-4f2e-8d7b-48e70a8a7929-kube-api-access-9zxxd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpzq8\" (UID: \"7c7df76b-e600-4f2e-8d7b-48e70a8a7929\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.351750 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.351675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:40:19.403026 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.402982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:19.424130 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.424102    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:40:19.547074 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.547039    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"]
Jul 15 13:40:19.550023 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:40:19.549996    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c7df76b_e600_4f2e_8d7b_48e70a8a7929.slice/crio-96b0b8e9762443ec0435e388f55f05fda143ef2de5d00c2cddcd1c16ad817bb5 WatchSource:0}: Error finding container 96b0b8e9762443ec0435e388f55f05fda143ef2de5d00c2cddcd1c16ad817bb5: Status 404 returned error can't find the container with id 96b0b8e9762443ec0435e388f55f05fda143ef2de5d00c2cddcd1c16ad817bb5
Jul 15 13:40:19.984080 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.984038    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8" event={"ID":"7c7df76b-e600-4f2e-8d7b-48e70a8a7929","Type":"ContainerStarted","Data":"940cc50988d7487104430003bbfa5e466fcc425aa073a5e6bb36ef9bafa98301"}
Jul 15 13:40:19.984252 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:19.984104    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8" event={"ID":"7c7df76b-e600-4f2e-8d7b-48e70a8a7929","Type":"ContainerStarted","Data":"96b0b8e9762443ec0435e388f55f05fda143ef2de5d00c2cddcd1c16ad817bb5"}
Jul 15 13:40:20.007455 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:20.007389    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8" podStartSLOduration=0.816969736 podStartE2EDuration="1.007369007s" totalImagesPullingTime="190.399271ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:40:20.006787207 +0000 UTC m=+664.349722297" watchObservedRunningTime="2026-07-15 13:40:20.007369007 +0000 UTC m=+664.350304093"
Jul 15 13:40:29.403644 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:29.403587    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:29.404044 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:29.403683    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:29.416383 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:29.416352    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:30.038583 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:30.038552    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpzq8"
Jul 15 13:40:30.304806 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:30.304727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:40:40.072155 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:40.072122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:40:40.174568 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:40.174537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:40:49.417463 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:40:49.417429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:41:45.103070 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:41:45.103032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:41:45.258299 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:41:45.258235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:41:46.316115 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:41:46.315964    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:41:46.460799 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:41:46.460749    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:42:04.850869 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:42:04.850830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:42:04.952693 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:42:04.952662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:42:47.275919 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:42:47.275882    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:42:47.355588 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:42:47.355559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:43:09.820342 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:09.820300    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:43:09.879662 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:09.879623    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577b8d5d55-ktvhg"]
Jul 15 13:43:09.879923 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:09.879880    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-577b8d5d55-ktvhg" podUID="3bc7608d-ada9-4c0b-a36e-e501a76136df" containerName="manager" containerID="cri-o://fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc" gracePeriod=30
Jul 15 13:43:10.130989 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.130955    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:43:10.254275 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.254236    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6htfz\" (UniqueName: \"kubernetes.io/projected/3bc7608d-ada9-4c0b-a36e-e501a76136df-kube-api-access-6htfz\") pod \"3bc7608d-ada9-4c0b-a36e-e501a76136df\" (UID: \"3bc7608d-ada9-4c0b-a36e-e501a76136df\") "
Jul 15 13:43:10.254447 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.254330    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bc7608d-ada9-4c0b-a36e-e501a76136df-webhook-cert\") pod \"3bc7608d-ada9-4c0b-a36e-e501a76136df\" (UID: \"3bc7608d-ada9-4c0b-a36e-e501a76136df\") "
Jul 15 13:43:10.256519 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.256482    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bc7608d-ada9-4c0b-a36e-e501a76136df-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3bc7608d-ada9-4c0b-a36e-e501a76136df" (UID: "3bc7608d-ada9-4c0b-a36e-e501a76136df"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:43:10.256640 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.256541    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bc7608d-ada9-4c0b-a36e-e501a76136df-kube-api-access-6htfz" (OuterVolumeSpecName: "kube-api-access-6htfz") pod "3bc7608d-ada9-4c0b-a36e-e501a76136df" (UID: "3bc7608d-ada9-4c0b-a36e-e501a76136df"). InnerVolumeSpecName "kube-api-access-6htfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:43:10.355359 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.355321    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6htfz\" (UniqueName: \"kubernetes.io/projected/3bc7608d-ada9-4c0b-a36e-e501a76136df-kube-api-access-6htfz\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:43:10.355359 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.355351    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3bc7608d-ada9-4c0b-a36e-e501a76136df-webhook-cert\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:43:10.712958 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.712922    2577 generic.go:356] "Generic (PLEG): container finished" podID="3bc7608d-ada9-4c0b-a36e-e501a76136df" containerID="fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc" exitCode=0
Jul 15 13:43:10.713140 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.712979    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577b8d5d55-ktvhg"
Jul 15 13:43:10.713140 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.712980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577b8d5d55-ktvhg" event={"ID":"3bc7608d-ada9-4c0b-a36e-e501a76136df","Type":"ContainerDied","Data":"fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc"}
Jul 15 13:43:10.713140 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.713124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577b8d5d55-ktvhg" event={"ID":"3bc7608d-ada9-4c0b-a36e-e501a76136df","Type":"ContainerDied","Data":"3f7fd0c3483c5cc19fc63959f66322e75619b564e644e5aac5678de9e624e439"}
Jul 15 13:43:10.713244 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.713147    2577 scope.go:117] "RemoveContainer" containerID="fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc"
Jul 15 13:43:10.722300 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.722253    2577 scope.go:117] "RemoveContainer" containerID="fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc"
Jul 15 13:43:10.722602 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:43:10.722583    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc\": container with ID starting with fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc not found: ID does not exist" containerID="fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc"
Jul 15 13:43:10.722667 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.722616    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc"} err="failed to get container status \"fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc\": rpc error: code = NotFound desc = could not find container \"fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc\": container with ID starting with fd8da887b22cd84af6816b6ae60829fc8adfb778ae82474bdd1a0108da64fbbc not found: ID does not exist"
Jul 15 13:43:10.738112 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.738080    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577b8d5d55-ktvhg"]
Jul 15 13:43:10.749522 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:10.749494    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-577b8d5d55-ktvhg"]
Jul 15 13:43:11.501683 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.501650    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:43:11.639101 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.639070    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-577b8d5d55-bz7rm"]
Jul 15 13:43:11.639481 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.639467    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bc7608d-ada9-4c0b-a36e-e501a76136df" containerName="manager"
Jul 15 13:43:11.639538 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.639482    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bc7608d-ada9-4c0b-a36e-e501a76136df" containerName="manager"
Jul 15 13:43:11.639580 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.639544    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bc7608d-ada9-4c0b-a36e-e501a76136df" containerName="manager"
Jul 15 13:43:11.643628 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.643609    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577b8d5d55-bz7rm"
Jul 15 13:43:11.649088 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.649042    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 13:43:11.649088 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.649054    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bphdb\""
Jul 15 13:43:11.649088 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.649047    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 13:43:11.657550 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.657527    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577b8d5d55-bz7rm"]
Jul 15 13:43:11.767458 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.767421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ssrm\" (UniqueName: \"kubernetes.io/projected/de9faf9e-d48b-4834-8a68-292bb362d81f-kube-api-access-4ssrm\") pod \"maas-controller-577b8d5d55-bz7rm\" (UID: \"de9faf9e-d48b-4834-8a68-292bb362d81f\") " pod="opendatahub/maas-controller-577b8d5d55-bz7rm"
Jul 15 13:43:11.767458 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.767463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de9faf9e-d48b-4834-8a68-292bb362d81f-webhook-cert\") pod \"maas-controller-577b8d5d55-bz7rm\" (UID: \"de9faf9e-d48b-4834-8a68-292bb362d81f\") " pod="opendatahub/maas-controller-577b8d5d55-bz7rm"
Jul 15 13:43:11.868505 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.868467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ssrm\" (UniqueName: \"kubernetes.io/projected/de9faf9e-d48b-4834-8a68-292bb362d81f-kube-api-access-4ssrm\") pod \"maas-controller-577b8d5d55-bz7rm\" (UID: \"de9faf9e-d48b-4834-8a68-292bb362d81f\") " pod="opendatahub/maas-controller-577b8d5d55-bz7rm"
Jul 15 13:43:11.868650 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.868513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de9faf9e-d48b-4834-8a68-292bb362d81f-webhook-cert\") pod \"maas-controller-577b8d5d55-bz7rm\" (UID: \"de9faf9e-d48b-4834-8a68-292bb362d81f\") " pod="opendatahub/maas-controller-577b8d5d55-bz7rm"
Jul 15 13:43:11.871212 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.871182    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de9faf9e-d48b-4834-8a68-292bb362d81f-webhook-cert\") pod \"maas-controller-577b8d5d55-bz7rm\" (UID: \"de9faf9e-d48b-4834-8a68-292bb362d81f\") " pod="opendatahub/maas-controller-577b8d5d55-bz7rm"
Jul 15 13:43:11.881812 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.881784    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ssrm\" (UniqueName: \"kubernetes.io/projected/de9faf9e-d48b-4834-8a68-292bb362d81f-kube-api-access-4ssrm\") pod \"maas-controller-577b8d5d55-bz7rm\" (UID: \"de9faf9e-d48b-4834-8a68-292bb362d81f\") " pod="opendatahub/maas-controller-577b8d5d55-bz7rm"
Jul 15 13:43:11.953735 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:11.953690    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577b8d5d55-bz7rm"
Jul 15 13:43:12.097602 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:12.097574    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577b8d5d55-bz7rm"]
Jul 15 13:43:12.147796 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:12.147763    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bc7608d-ada9-4c0b-a36e-e501a76136df" path="/var/lib/kubelet/pods/3bc7608d-ada9-4c0b-a36e-e501a76136df/volumes"
Jul 15 13:43:12.725439 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:12.725407    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577b8d5d55-bz7rm" event={"ID":"de9faf9e-d48b-4834-8a68-292bb362d81f","Type":"ContainerStarted","Data":"c65042994b4d96206f210ebccd3db43873009b60bc570df38a0b5cc5948ddf76"}
Jul 15 13:43:13.732004 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:13.731971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577b8d5d55-bz7rm" event={"ID":"de9faf9e-d48b-4834-8a68-292bb362d81f","Type":"ContainerStarted","Data":"68bbb76a6b8c9417bbfc797845ea8a01cc0b4e3c6c5f91545ec259b67470cb37"}
Jul 15 13:43:13.732521 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:13.732091    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-577b8d5d55-bz7rm"
Jul 15 13:43:13.757384 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:13.757337    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-577b8d5d55-bz7rm" podStartSLOduration=2.185492312 podStartE2EDuration="2.757321659s" totalImagesPullingTime="571.829347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:43:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:43:13.756325884 +0000 UTC m=+838.099260988" watchObservedRunningTime="2026-07-15 13:43:13.757321659 +0000 UTC m=+838.100256738"
Jul 15 13:43:24.743943 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:24.743913    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-577b8d5d55-bz7rm"
Jul 15 13:43:53.471324 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:53.471287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:43:53.570645 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:53.570608    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:43:53.693371 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:43:53.693337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:44:01.100507 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:44:01.100472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:44:01.217733 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:44:01.217677    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:44:01.330614 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:44:01.330582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:44:01.562361 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:44:01.562332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:44:07.963247 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:44:07.963161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:44:08.024288 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:44:08.024240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:44:16.090931 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:44:16.090900    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:44:16.093215 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:44:16.093193    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:45:00.153887 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:00.153849    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq"]
Jul 15 13:45:00.157089 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:00.157072    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq"
Jul 15 13:45:00.160072 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:00.160049    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 13:45:00.166415 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:00.166394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq"
Jul 15 13:45:00.176644 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:00.176604    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq"]
Jul 15 13:45:00.513892 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:00.513865    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq"]
Jul 15 13:45:00.516458 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:45:00.516429    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c010e1_c41f_4a27_bd85_3f50bfaa1dc7.slice/crio-7a471ca8ba3e17c75df7fda842ff1afb73c6d85811a60a56fca2b213e843caa8 WatchSource:0}: Error finding container 7a471ca8ba3e17c75df7fda842ff1afb73c6d85811a60a56fca2b213e843caa8: Status 404 returned error can't find the container with id 7a471ca8ba3e17c75df7fda842ff1afb73c6d85811a60a56fca2b213e843caa8
Jul 15 13:45:00.518388 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:00.518369    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 13:45:01.216187 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:01.216151    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq" event={"ID":"f8c010e1-c41f-4a27-bd85-3f50bfaa1dc7","Type":"ContainerStarted","Data":"7a471ca8ba3e17c75df7fda842ff1afb73c6d85811a60a56fca2b213e843caa8"}
Jul 15 13:45:04.232102 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:04.232019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq" event={"ID":"f8c010e1-c41f-4a27-bd85-3f50bfaa1dc7","Type":"ContainerStarted","Data":"e5af10a423a1741e718e1188d87e256e2589945d80797a7c77bc29140ee5fd81"}
Jul 15 13:45:04.249883 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:04.249832    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq" podStartSLOduration=1.232510002 podStartE2EDuration="4.249815539s" totalImagesPullingTime="3.017305537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:45:04.248427815 +0000 UTC m=+948.591362894" watchObservedRunningTime="2026-07-15 13:45:04.249815539 +0000 UTC m=+948.592750619"
Jul 15 13:45:05.237757 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:05.237721    2577 generic.go:356] "Generic (PLEG): container finished" podID="f8c010e1-c41f-4a27-bd85-3f50bfaa1dc7" containerID="e5af10a423a1741e718e1188d87e256e2589945d80797a7c77bc29140ee5fd81" exitCode=0
Jul 15 13:45:05.238136 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:05.237814    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq" event={"ID":"f8c010e1-c41f-4a27-bd85-3f50bfaa1dc7","Type":"ContainerDied","Data":"e5af10a423a1741e718e1188d87e256e2589945d80797a7c77bc29140ee5fd81"}
Jul 15 13:45:06.380157 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:06.380134    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq"
Jul 15 13:45:07.247788 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:07.247756    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq"
Jul 15 13:45:07.247788 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:07.247781    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735385-xkmrq" event={"ID":"f8c010e1-c41f-4a27-bd85-3f50bfaa1dc7","Type":"ContainerDied","Data":"7a471ca8ba3e17c75df7fda842ff1afb73c6d85811a60a56fca2b213e843caa8"}
Jul 15 13:45:07.247995 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:45:07.247825    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a471ca8ba3e17c75df7fda842ff1afb73c6d85811a60a56fca2b213e843caa8"
Jul 15 13:46:28.309641 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:46:28.309604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:46:29.807739 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:46:29.807705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:49:07.957521 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:49:07.957487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:49:16.120400 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:49:16.120369    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:49:16.120969 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:49:16.120953    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:49:28.830072 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:49:28.830032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:49:30.778374 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:49:30.778343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:50:35.327989 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:50:35.327902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:54:16.145888 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:54:16.145845    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:54:16.148553 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:54:16.148532    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:54:57.423410 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:54:57.423379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:25.632101 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:25.632064    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:25.741991 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:25.741962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:26.173550 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:26.173513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:29.130222 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:29.130183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:29.225669 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:29.225629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:32.314489 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:32.314455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:32.417754 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:32.417721    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:32.498492 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:32.498451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:33.605825 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.605781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:33.734818 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.734785    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:33.806354 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.806320    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"]
Jul 15 13:55:33.806672 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.806659    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8c010e1-c41f-4a27-bd85-3f50bfaa1dc7" containerName="cleanup"
Jul 15 13:55:33.806672 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.806673    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c010e1-c41f-4a27-bd85-3f50bfaa1dc7" containerName="cleanup"
Jul 15 13:55:33.806756 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.806735    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8c010e1-c41f-4a27-bd85-3f50bfaa1dc7" containerName="cleanup"
Jul 15 13:55:33.809714 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.809696    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:33.813339 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.813311    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-044bd9bf-serving-cert\""
Jul 15 13:55:33.836923 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.836895    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"]
Jul 15 13:55:33.857704 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.857629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:33.926597 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.926560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-maas-api-tls\") pod \"maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:33.926805 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.926622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-openshift-service-ca\") pod \"maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:33.926805 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:33.926674    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxq5q\" (UniqueName: \"kubernetes.io/projected/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-kube-api-access-rxq5q\") pod \"maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:34.027247 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.027207    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-openshift-service-ca\") pod \"maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:34.027247 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.027250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxq5q\" (UniqueName: \"kubernetes.io/projected/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-kube-api-access-rxq5q\") pod \"maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:34.027516 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.027355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-maas-api-tls\") pod \"maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:34.027516 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:55:34.027457    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-serving-cert: secret "maas-api-e2e-ait-044bd9bf-serving-cert" not found
Jul 15 13:55:34.027602 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:55:34.027520    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-maas-api-tls podName:ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd nodeName:}" failed. No retries permitted until 2026-07-15 13:55:34.527504369 +0000 UTC m=+1578.870439426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-maas-api-tls") pod "maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h" (UID: "ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd") : secret "maas-api-e2e-ait-044bd9bf-serving-cert" not found
Jul 15 13:55:34.027868 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.027848    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-openshift-service-ca\") pod \"maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:34.039129 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.039094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxq5q\" (UniqueName: \"kubernetes.io/projected/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-kube-api-access-rxq5q\") pod \"maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:34.132157 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.132078    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:34.443082 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.443001    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:34.532911 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.532871    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-maas-api-tls\") pod \"maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:34.535450 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.535427    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-maas-api-tls\") pod \"maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:34.545234 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.545208    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:34.720630 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.720540    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:34.860433 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.860402    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"]
Jul 15 13:55:34.864245 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:55:34.864218    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef3a2bfd_5e0c_49c4_869f_7f7fb97be0bd.slice/crio-0d0b15333e95ab75f377c0acc812b1c728bcdaa82bbd56cb0ffb4083d65b7d77 WatchSource:0}: Error finding container 0d0b15333e95ab75f377c0acc812b1c728bcdaa82bbd56cb0ffb4083d65b7d77: Status 404 returned error can't find the container with id 0d0b15333e95ab75f377c0acc812b1c728bcdaa82bbd56cb0ffb4083d65b7d77
Jul 15 13:55:34.865620 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:34.865601    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 13:55:35.040928 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:35.040892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h" event={"ID":"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd","Type":"ContainerStarted","Data":"0d0b15333e95ab75f377c0acc812b1c728bcdaa82bbd56cb0ffb4083d65b7d77"}
Jul 15 13:55:36.046534 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:36.046498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h" event={"ID":"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd","Type":"ContainerStarted","Data":"ff49bc699a1868b151ded4910905dc033e1f09144fc3eb220eed6fc0ab3e2d3d"}
Jul 15 13:55:36.046938 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:36.046618    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:36.067035 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:36.066986    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h" podStartSLOduration=2.615931416 podStartE2EDuration="3.066969669s" totalImagesPullingTime="451.038253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:55:36.065102352 +0000 UTC m=+1580.408037453" watchObservedRunningTime="2026-07-15 13:55:36.066969669 +0000 UTC m=+1580.409904749"
Jul 15 13:55:36.894325 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:36.894257    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:38.726822 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:38.726785    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"]
Jul 15 13:55:38.730232 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:38.730214    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"
Jul 15 13:55:38.732985 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:38.732960    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 13:55:38.738633 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:38.738609    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"]
Jul 15 13:55:38.867358 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:38.867326    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a9a1b9-8487-40cd-af4e-ea904afe74b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd\" (UID: \"35a9a1b9-8487-40cd-af4e-ea904afe74b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"
Jul 15 13:55:38.968823 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:38.968781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a9a1b9-8487-40cd-af4e-ea904afe74b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd\" (UID: \"35a9a1b9-8487-40cd-af4e-ea904afe74b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"
Jul 15 13:55:38.969510 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:38.969490    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a9a1b9-8487-40cd-af4e-ea904afe74b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd\" (UID: \"35a9a1b9-8487-40cd-af4e-ea904afe74b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"
Jul 15 13:55:39.041436 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:39.041399    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"
Jul 15 13:55:39.180884 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:39.180811    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"]
Jul 15 13:55:39.183466 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:55:39.183437    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a9a1b9_8487_40cd_af4e_ea904afe74b0.slice/crio-271bdcd54caa1faf515fcf05c3912104903acebfe80dc94f73badc6d98898f64 WatchSource:0}: Error finding container 271bdcd54caa1faf515fcf05c3912104903acebfe80dc94f73badc6d98898f64: Status 404 returned error can't find the container with id 271bdcd54caa1faf515fcf05c3912104903acebfe80dc94f73badc6d98898f64
Jul 15 13:55:40.066703 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:40.066619    2577 generic.go:356] "Generic (PLEG): container finished" podID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerID="a353e7a5bc7844d3d34d15ae166c628111022a583d12275612c5155dc6bab46c" exitCode=7
Jul 15 13:55:40.067080 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:40.066704    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd" event={"ID":"35a9a1b9-8487-40cd-af4e-ea904afe74b0","Type":"ContainerDied","Data":"a353e7a5bc7844d3d34d15ae166c628111022a583d12275612c5155dc6bab46c"}
Jul 15 13:55:40.067080 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:40.066768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd" event={"ID":"35a9a1b9-8487-40cd-af4e-ea904afe74b0","Type":"ContainerStarted","Data":"271bdcd54caa1faf515fcf05c3912104903acebfe80dc94f73badc6d98898f64"}
Jul 15 13:55:40.067080 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:40.066852    2577 scope.go:117] "RemoveContainer" containerID="a353e7a5bc7844d3d34d15ae166c628111022a583d12275612c5155dc6bab46c"
Jul 15 13:55:41.072798 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:41.072703    2577 generic.go:356] "Generic (PLEG): container finished" podID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerID="8604de91adb21fe0d8d5439aa4c81c1b675e746a40d310f686efe08cdbb8f067" exitCode=7
Jul 15 13:55:41.072798 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:41.072777    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd" event={"ID":"35a9a1b9-8487-40cd-af4e-ea904afe74b0","Type":"ContainerDied","Data":"8604de91adb21fe0d8d5439aa4c81c1b675e746a40d310f686efe08cdbb8f067"}
Jul 15 13:55:41.073245 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:41.072835    2577 scope.go:117] "RemoveContainer" containerID="a353e7a5bc7844d3d34d15ae166c628111022a583d12275612c5155dc6bab46c"
Jul 15 13:55:41.073245 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:41.073015    2577 scope.go:117] "RemoveContainer" containerID="8604de91adb21fe0d8d5439aa4c81c1b675e746a40d310f686efe08cdbb8f067"
Jul 15 13:55:41.073245 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:55:41.073227    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd_odh-ai-gateway-infra(35a9a1b9-8487-40cd-af4e-ea904afe74b0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0"
Jul 15 13:55:42.057320 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:42.057287    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:55:42.079150 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:42.079122    2577 scope.go:117] "RemoveContainer" containerID="8604de91adb21fe0d8d5439aa4c81c1b675e746a40d310f686efe08cdbb8f067"
Jul 15 13:55:42.079646 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:55:42.079416    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd_odh-ai-gateway-infra(35a9a1b9-8487-40cd-af4e-ea904afe74b0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0"
Jul 15 13:55:56.143525 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:56.143488    2577 scope.go:117] "RemoveContainer" containerID="8604de91adb21fe0d8d5439aa4c81c1b675e746a40d310f686efe08cdbb8f067"
Jul 15 13:55:57.144654 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:57.144560    2577 generic.go:356] "Generic (PLEG): container finished" podID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerID="a47a2cf3950dccab011554517e680219d78fc3da981180df0a00a83fb8895f36" exitCode=0
Jul 15 13:55:57.144654 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:57.144628    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd" event={"ID":"35a9a1b9-8487-40cd-af4e-ea904afe74b0","Type":"ContainerDied","Data":"a47a2cf3950dccab011554517e680219d78fc3da981180df0a00a83fb8895f36"}
Jul 15 13:55:57.145098 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:57.144692    2577 scope.go:117] "RemoveContainer" containerID="8604de91adb21fe0d8d5439aa4c81c1b675e746a40d310f686efe08cdbb8f067"
Jul 15 13:55:58.178525 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:58.178487    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"]
Jul 15 13:55:58.296278 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:58.296229    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"
Jul 15 13:55:58.340185 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:58.340152    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a9a1b9-8487-40cd-af4e-ea904afe74b0-maas-api-service-ca\") pod \"35a9a1b9-8487-40cd-af4e-ea904afe74b0\" (UID: \"35a9a1b9-8487-40cd-af4e-ea904afe74b0\") "
Jul 15 13:55:58.340569 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:58.340541    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35a9a1b9-8487-40cd-af4e-ea904afe74b0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "35a9a1b9-8487-40cd-af4e-ea904afe74b0" (UID: "35a9a1b9-8487-40cd-af4e-ea904afe74b0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:55:58.441147 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:58.441045    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35a9a1b9-8487-40cd-af4e-ea904afe74b0-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:55:59.157874 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:59.157838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd" event={"ID":"35a9a1b9-8487-40cd-af4e-ea904afe74b0","Type":"ContainerDied","Data":"271bdcd54caa1faf515fcf05c3912104903acebfe80dc94f73badc6d98898f64"}
Jul 15 13:55:59.158113 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:59.157893    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="271bdcd54caa1faf515fcf05c3912104903acebfe80dc94f73badc6d98898f64"
Jul 15 13:55:59.158113 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:59.157857    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"
Jul 15 13:55:59.194895 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:59.194861    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"]
Jul 15 13:55:59.198900 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:59.198875    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-jm5vd"]
Jul 15 13:55:59.851583 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:59.851552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:55:59.951584 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:55:59.951550    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:56:00.146988 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:00.146908    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" path="/var/lib/kubelet/pods/35a9a1b9-8487-40cd-af4e-ea904afe74b0/volumes"
Jul 15 13:56:38.785082 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.784984    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw"]
Jul 15 13:56:38.785575 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.785553    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerName="revoke-keys"
Jul 15 13:56:38.785575 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.785573    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerName="revoke-keys"
Jul 15 13:56:38.785647 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.785603    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerName="revoke-keys"
Jul 15 13:56:38.785647 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.785612    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerName="revoke-keys"
Jul 15 13:56:38.785647 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.785624    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerName="revoke-keys"
Jul 15 13:56:38.785647 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.785632    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerName="revoke-keys"
Jul 15 13:56:38.785767 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.785712    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerName="revoke-keys"
Jul 15 13:56:38.785767 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.785724    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerName="revoke-keys"
Jul 15 13:56:38.785767 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.785733    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="35a9a1b9-8487-40cd-af4e-ea904afe74b0" containerName="revoke-keys"
Jul 15 13:56:38.790314 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.790287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw"
Jul 15 13:56:38.793836 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.793804    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 13:56:38.798744 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.798718    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw"]
Jul 15 13:56:38.902629 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:38.902593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb0fe70c-0cf9-421b-84c0-c84998cb25cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-044bd9bf-242nw\" (UID: \"fb0fe70c-0cf9-421b-84c0-c84998cb25cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw"
Jul 15 13:56:39.003303 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:39.003248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb0fe70c-0cf9-421b-84c0-c84998cb25cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-044bd9bf-242nw\" (UID: \"fb0fe70c-0cf9-421b-84c0-c84998cb25cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw"
Jul 15 13:56:39.003897 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:39.003878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb0fe70c-0cf9-421b-84c0-c84998cb25cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-044bd9bf-242nw\" (UID: \"fb0fe70c-0cf9-421b-84c0-c84998cb25cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw"
Jul 15 13:56:39.104447 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:39.104361    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw"
Jul 15 13:56:39.241896 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:39.241739    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw"]
Jul 15 13:56:39.244598 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:56:39.244563    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0fe70c_0cf9_421b_84c0_c84998cb25cc.slice/crio-2d3953828359e38032f9fbeaa50febe57fa0de8fca47a7cd3c7a4b1215994046 WatchSource:0}: Error finding container 2d3953828359e38032f9fbeaa50febe57fa0de8fca47a7cd3c7a4b1215994046: Status 404 returned error can't find the container with id 2d3953828359e38032f9fbeaa50febe57fa0de8fca47a7cd3c7a4b1215994046
Jul 15 13:56:39.342120 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:39.342083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw" event={"ID":"fb0fe70c-0cf9-421b-84c0-c84998cb25cc","Type":"ContainerStarted","Data":"2d3953828359e38032f9fbeaa50febe57fa0de8fca47a7cd3c7a4b1215994046"}
Jul 15 13:56:40.348737 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:40.348698    2577 generic.go:356] "Generic (PLEG): container finished" podID="fb0fe70c-0cf9-421b-84c0-c84998cb25cc" containerID="c9237ec41ca4e82f5d57734e5865afc8ce03b365e00979c469c17d8fbf6d3435" exitCode=0
Jul 15 13:56:40.349138 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:40.348775    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw" event={"ID":"fb0fe70c-0cf9-421b-84c0-c84998cb25cc","Type":"ContainerDied","Data":"c9237ec41ca4e82f5d57734e5865afc8ce03b365e00979c469c17d8fbf6d3435"}
Jul 15 13:56:41.490078 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:41.489973    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw"
Jul 15 13:56:41.520323 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:41.520296    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb0fe70c-0cf9-421b-84c0-c84998cb25cc-maas-api-service-ca\") pod \"fb0fe70c-0cf9-421b-84c0-c84998cb25cc\" (UID: \"fb0fe70c-0cf9-421b-84c0-c84998cb25cc\") "
Jul 15 13:56:41.520617 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:41.520596    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb0fe70c-0cf9-421b-84c0-c84998cb25cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fb0fe70c-0cf9-421b-84c0-c84998cb25cc" (UID: "fb0fe70c-0cf9-421b-84c0-c84998cb25cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:56:41.621487 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:41.621403    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb0fe70c-0cf9-421b-84c0-c84998cb25cc-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:56:42.360511 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:42.360479    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw"
Jul 15 13:56:42.360511 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:42.360488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-044bd9bf-242nw" event={"ID":"fb0fe70c-0cf9-421b-84c0-c84998cb25cc","Type":"ContainerDied","Data":"2d3953828359e38032f9fbeaa50febe57fa0de8fca47a7cd3c7a4b1215994046"}
Jul 15 13:56:42.360706 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:42.360537    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d3953828359e38032f9fbeaa50febe57fa0de8fca47a7cd3c7a4b1215994046"
Jul 15 13:56:49.182429 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.182393    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"]
Jul 15 13:56:49.182815 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.182623    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h" podUID="ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd" containerName="maas-api" containerID="cri-o://ff49bc699a1868b151ded4910905dc033e1f09144fc3eb220eed6fc0ab3e2d3d" gracePeriod=30
Jul 15 13:56:49.227918 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.227887    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:56:49.392501 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.392467    2577 generic.go:356] "Generic (PLEG): container finished" podID="ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd" containerID="ff49bc699a1868b151ded4910905dc033e1f09144fc3eb220eed6fc0ab3e2d3d" exitCode=0
Jul 15 13:56:49.392669 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.392528    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h" event={"ID":"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd","Type":"ContainerDied","Data":"ff49bc699a1868b151ded4910905dc033e1f09144fc3eb220eed6fc0ab3e2d3d"}
Jul 15 13:56:49.428939 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.428910    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:56:49.489502 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.489407    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-openshift-service-ca\") pod \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") "
Jul 15 13:56:49.489502 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.489466    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-maas-api-tls\") pod \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") "
Jul 15 13:56:49.489502 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.489491    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxq5q\" (UniqueName: \"kubernetes.io/projected/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-kube-api-access-rxq5q\") pod \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\" (UID: \"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd\") "
Jul 15 13:56:49.489792 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.489774    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd" (UID: "ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:56:49.491956 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.491919    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd" (UID: "ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:56:49.492075 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.491953    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-kube-api-access-rxq5q" (OuterVolumeSpecName: "kube-api-access-rxq5q") pod "ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd" (UID: "ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd"). InnerVolumeSpecName "kube-api-access-rxq5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:56:49.590209 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.590170    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:56:49.590209 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.590206    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:56:49.590209 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:49.590215    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxq5q\" (UniqueName: \"kubernetes.io/projected/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd-kube-api-access-rxq5q\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:56:50.398814 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:50.398771    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h" event={"ID":"ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd","Type":"ContainerDied","Data":"0d0b15333e95ab75f377c0acc812b1c728bcdaa82bbd56cb0ffb4083d65b7d77"}
Jul 15 13:56:50.398814 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:50.398804    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"
Jul 15 13:56:50.399363 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:50.398859    2577 scope.go:117] "RemoveContainer" containerID="ff49bc699a1868b151ded4910905dc033e1f09144fc3eb220eed6fc0ab3e2d3d"
Jul 15 13:56:50.416518 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:50.416489    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"]
Jul 15 13:56:50.420340 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:50.420315    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-044bd9bf-c5fd4b587-xx48h"]
Jul 15 13:56:52.147106 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:52.147072    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd" path="/var/lib/kubelet/pods/ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd/volumes"
Jul 15 13:56:54.253499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:56:54.253464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:00.302662 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:00.302627    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:00.432858 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:00.432827    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:04.070672 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:04.070640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:04.481670 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:04.481579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:04.585680 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:04.585646    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:04.714855 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:04.714811    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:06.027618 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.027585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:06.249375 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.249327    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"]
Jul 15 13:57:06.249927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.249906    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb0fe70c-0cf9-421b-84c0-c84998cb25cc" containerName="revoke-keys"
Jul 15 13:57:06.249927 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.249925    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb0fe70c-0cf9-421b-84c0-c84998cb25cc" containerName="revoke-keys"
Jul 15 13:57:06.250136 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.249943    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd" containerName="maas-api"
Jul 15 13:57:06.250136 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.249952    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd" containerName="maas-api"
Jul 15 13:57:06.250136 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.250042    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb0fe70c-0cf9-421b-84c0-c84998cb25cc" containerName="revoke-keys"
Jul 15 13:57:06.250136 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.250056    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef3a2bfd-5e0c-49c4-869f-7f7fb97be0bd" containerName="maas-api"
Jul 15 13:57:06.255193 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.255159    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.257828 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.257806    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-65951df1-serving-cert\""
Jul 15 13:57:06.264852 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.264821    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"]
Jul 15 13:57:06.292695 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.292612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:06.336464 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.336427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99f4710d-254a-4dec-862b-0fa167a7ba39-maas-api-tls\") pod \"maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.336608 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.336493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bws69\" (UniqueName: \"kubernetes.io/projected/99f4710d-254a-4dec-862b-0fa167a7ba39-kube-api-access-bws69\") pod \"maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.336608 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.336549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99f4710d-254a-4dec-862b-0fa167a7ba39-openshift-service-ca\") pod \"maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.437118 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.437082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bws69\" (UniqueName: \"kubernetes.io/projected/99f4710d-254a-4dec-862b-0fa167a7ba39-kube-api-access-bws69\") pod \"maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.437318 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.437129    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99f4710d-254a-4dec-862b-0fa167a7ba39-openshift-service-ca\") pod \"maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.437318 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.437179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99f4710d-254a-4dec-862b-0fa167a7ba39-maas-api-tls\") pod \"maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.437396 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:57:06.437320    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-serving-cert: secret "maas-api-e2e-migrate-65951df1-serving-cert" not found
Jul 15 13:57:06.437438 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:57:06.437400    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99f4710d-254a-4dec-862b-0fa167a7ba39-maas-api-tls podName:99f4710d-254a-4dec-862b-0fa167a7ba39 nodeName:}" failed. No retries permitted until 2026-07-15 13:57:06.93737953 +0000 UTC m=+1671.280314589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99f4710d-254a-4dec-862b-0fa167a7ba39-maas-api-tls") pod "maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4" (UID: "99f4710d-254a-4dec-862b-0fa167a7ba39") : secret "maas-api-e2e-migrate-65951df1-serving-cert" not found
Jul 15 13:57:06.437894 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.437874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99f4710d-254a-4dec-862b-0fa167a7ba39-openshift-service-ca\") pod \"maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.447510 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.447485    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bws69\" (UniqueName: \"kubernetes.io/projected/99f4710d-254a-4dec-862b-0fa167a7ba39-kube-api-access-bws69\") pod \"maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.528926 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.528897    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:06.858030 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.857999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:06.942065 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.942028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99f4710d-254a-4dec-862b-0fa167a7ba39-maas-api-tls\") pod \"maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.944617 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.944592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99f4710d-254a-4dec-862b-0fa167a7ba39-maas-api-tls\") pod \"maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:06.953682 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:06.953656    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:07.166924 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:07.166830    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:07.313488 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:07.313461    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"]
Jul 15 13:57:07.316436 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:57:07.316395    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99f4710d_254a_4dec_862b_0fa167a7ba39.slice/crio-1118a29ad9a6463a2db7a4695d9a7d40100c35582e1a9bda067d930e072c92ae WatchSource:0}: Error finding container 1118a29ad9a6463a2db7a4695d9a7d40100c35582e1a9bda067d930e072c92ae: Status 404 returned error can't find the container with id 1118a29ad9a6463a2db7a4695d9a7d40100c35582e1a9bda067d930e072c92ae
Jul 15 13:57:07.479384 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:07.479300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4" event={"ID":"99f4710d-254a-4dec-862b-0fa167a7ba39","Type":"ContainerStarted","Data":"1118a29ad9a6463a2db7a4695d9a7d40100c35582e1a9bda067d930e072c92ae"}
Jul 15 13:57:08.485210 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:08.485114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4" event={"ID":"99f4710d-254a-4dec-862b-0fa167a7ba39","Type":"ContainerStarted","Data":"df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8"}
Jul 15 13:57:08.485210 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:08.485194    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:08.508663 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:08.508602    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4" podStartSLOduration=1.708876444 podStartE2EDuration="2.508582565s" totalImagesPullingTime="799.706121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:57:08.507724431 +0000 UTC m=+1672.850659512" watchObservedRunningTime="2026-07-15 13:57:08.508582565 +0000 UTC m=+1672.851517644"
Jul 15 13:57:10.518401 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:10.518361    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"]
Jul 15 13:57:10.522007 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:10.521989    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"
Jul 15 13:57:10.526486 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:10.526458    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 13:57:10.534342 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:10.534312    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"]
Jul 15 13:57:10.674753 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:10.674705    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdefd2ee-5b80-422b-ac17-1214dafbaf7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-65951df1-npnlz\" (UID: \"fdefd2ee-5b80-422b-ac17-1214dafbaf7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"
Jul 15 13:57:10.775887 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:10.775802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdefd2ee-5b80-422b-ac17-1214dafbaf7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-65951df1-npnlz\" (UID: \"fdefd2ee-5b80-422b-ac17-1214dafbaf7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"
Jul 15 13:57:10.776456 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:10.776434    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdefd2ee-5b80-422b-ac17-1214dafbaf7f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-65951df1-npnlz\" (UID: \"fdefd2ee-5b80-422b-ac17-1214dafbaf7f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"
Jul 15 13:57:10.832410 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:10.832372    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"
Jul 15 13:57:10.976416 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:10.976382    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"]
Jul 15 13:57:10.979598 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:57:10.979566    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdefd2ee_5b80_422b_ac17_1214dafbaf7f.slice/crio-bf408e6750c69443d5852eeecbecca93aa5742cf13610d96bad3ae379dc7f698 WatchSource:0}: Error finding container bf408e6750c69443d5852eeecbecca93aa5742cf13610d96bad3ae379dc7f698: Status 404 returned error can't find the container with id bf408e6750c69443d5852eeecbecca93aa5742cf13610d96bad3ae379dc7f698
Jul 15 13:57:11.501805 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:11.501705    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz" event={"ID":"fdefd2ee-5b80-422b-ac17-1214dafbaf7f","Type":"ContainerStarted","Data":"7315fe13bca2dbaeb8abe0f62a40049a06336954f2fd5e56791dfa33b24de544"}
Jul 15 13:57:11.501805 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:11.501773    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz" event={"ID":"fdefd2ee-5b80-422b-ac17-1214dafbaf7f","Type":"ContainerStarted","Data":"bf408e6750c69443d5852eeecbecca93aa5742cf13610d96bad3ae379dc7f698"}
Jul 15 13:57:11.519526 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:11.519476    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz" podStartSLOduration=1.519460736 podStartE2EDuration="1.519460736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:57:11.517074209 +0000 UTC m=+1675.860009318" watchObservedRunningTime="2026-07-15 13:57:11.519460736 +0000 UTC m=+1675.862395849"
Jul 15 13:57:12.508224 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:12.508186    2577 generic.go:356] "Generic (PLEG): container finished" podID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerID="7315fe13bca2dbaeb8abe0f62a40049a06336954f2fd5e56791dfa33b24de544" exitCode=7
Jul 15 13:57:12.508449 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:12.508290    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz" event={"ID":"fdefd2ee-5b80-422b-ac17-1214dafbaf7f","Type":"ContainerDied","Data":"7315fe13bca2dbaeb8abe0f62a40049a06336954f2fd5e56791dfa33b24de544"}
Jul 15 13:57:12.508517 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:12.508483    2577 scope.go:117] "RemoveContainer" containerID="7315fe13bca2dbaeb8abe0f62a40049a06336954f2fd5e56791dfa33b24de544"
Jul 15 13:57:13.515370 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:13.515339    2577 generic.go:356] "Generic (PLEG): container finished" podID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerID="d115791d9fa5c8c7194a866d849ac45e7f0add17e922b1d5b5bf54182a319442" exitCode=7
Jul 15 13:57:13.515848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:13.515424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz" event={"ID":"fdefd2ee-5b80-422b-ac17-1214dafbaf7f","Type":"ContainerDied","Data":"d115791d9fa5c8c7194a866d849ac45e7f0add17e922b1d5b5bf54182a319442"}
Jul 15 13:57:13.515848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:13.515492    2577 scope.go:117] "RemoveContainer" containerID="7315fe13bca2dbaeb8abe0f62a40049a06336954f2fd5e56791dfa33b24de544"
Jul 15 13:57:13.515848 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:13.515729    2577 scope.go:117] "RemoveContainer" containerID="d115791d9fa5c8c7194a866d849ac45e7f0add17e922b1d5b5bf54182a319442"
Jul 15 13:57:13.515985 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:57:13.515965    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-65951df1-npnlz_odh-ai-gateway-infra(fdefd2ee-5b80-422b-ac17-1214dafbaf7f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f"
Jul 15 13:57:14.495211 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:14.495175    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:57:14.521990 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:14.521965    2577 scope.go:117] "RemoveContainer" containerID="d115791d9fa5c8c7194a866d849ac45e7f0add17e922b1d5b5bf54182a319442"
Jul 15 13:57:14.522442 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:57:14.522212    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-65951df1-npnlz_odh-ai-gateway-infra(fdefd2ee-5b80-422b-ac17-1214dafbaf7f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f"
Jul 15 13:57:15.627215 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:15.627189    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:15.726695 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:15.726653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:19.349998 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:19.349962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:20.492004 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:20.491967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:57:29.141015 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:29.140982    2577 scope.go:117] "RemoveContainer" containerID="d115791d9fa5c8c7194a866d849ac45e7f0add17e922b1d5b5bf54182a319442"
Jul 15 13:57:29.600177 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:29.600138    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz" event={"ID":"fdefd2ee-5b80-422b-ac17-1214dafbaf7f","Type":"ContainerStarted","Data":"1d280b81f5180ec2f2436fa6560b5923833d381efa944f1e8c4b366932a92cfc"}
Jul 15 13:57:30.606788 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:30.606757    2577 generic.go:356] "Generic (PLEG): container finished" podID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerID="1d280b81f5180ec2f2436fa6560b5923833d381efa944f1e8c4b366932a92cfc" exitCode=0
Jul 15 13:57:30.607279 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:30.606836    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz" event={"ID":"fdefd2ee-5b80-422b-ac17-1214dafbaf7f","Type":"ContainerDied","Data":"1d280b81f5180ec2f2436fa6560b5923833d381efa944f1e8c4b366932a92cfc"}
Jul 15 13:57:30.607279 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:30.606929    2577 scope.go:117] "RemoveContainer" containerID="d115791d9fa5c8c7194a866d849ac45e7f0add17e922b1d5b5bf54182a319442"
Jul 15 13:57:30.632796 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:30.632765    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"]
Jul 15 13:57:31.756801 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:31.756772    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"
Jul 15 13:57:31.868100 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:31.868006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdefd2ee-5b80-422b-ac17-1214dafbaf7f-maas-api-service-ca\") pod \"fdefd2ee-5b80-422b-ac17-1214dafbaf7f\" (UID: \"fdefd2ee-5b80-422b-ac17-1214dafbaf7f\") "
Jul 15 13:57:31.868411 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:31.868386    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdefd2ee-5b80-422b-ac17-1214dafbaf7f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fdefd2ee-5b80-422b-ac17-1214dafbaf7f" (UID: "fdefd2ee-5b80-422b-ac17-1214dafbaf7f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:57:31.968627 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:31.968594    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdefd2ee-5b80-422b-ac17-1214dafbaf7f-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:57:32.618205 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:32.618176    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"
Jul 15 13:57:32.618205 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:32.618203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz" event={"ID":"fdefd2ee-5b80-422b-ac17-1214dafbaf7f","Type":"ContainerDied","Data":"bf408e6750c69443d5852eeecbecca93aa5742cf13610d96bad3ae379dc7f698"}
Jul 15 13:57:32.618464 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:32.618250    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf408e6750c69443d5852eeecbecca93aa5742cf13610d96bad3ae379dc7f698"
Jul 15 13:57:32.655413 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:32.655380    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"]
Jul 15 13:57:32.657460 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:32.657438    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-npnlz"]
Jul 15 13:57:34.147176 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:57:34.147132    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" path="/var/lib/kubelet/pods/fdefd2ee-5b80-422b-ac17-1214dafbaf7f/volumes"
Jul 15 13:58:10.553870 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.553834    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r"]
Jul 15 13:58:10.554297 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.554227    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerName="revoke-keys"
Jul 15 13:58:10.554297 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.554239    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerName="revoke-keys"
Jul 15 13:58:10.554297 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.554249    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerName="revoke-keys"
Jul 15 13:58:10.554297 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.554255    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerName="revoke-keys"
Jul 15 13:58:10.554431 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.554341    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerName="revoke-keys"
Jul 15 13:58:10.554431 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.554350    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerName="revoke-keys"
Jul 15 13:58:10.557459 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.557440    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r"
Jul 15 13:58:10.560424 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.560400    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 13:58:10.567220 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.567197    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r"]
Jul 15 13:58:10.649593 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.649565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:10.661705 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.661664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036f5ce0-df20-4e4f-b7aa-9afde4618b1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r\" (UID: \"036f5ce0-df20-4e4f-b7aa-9afde4618b1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r"
Jul 15 13:58:10.762118 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.762082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036f5ce0-df20-4e4f-b7aa-9afde4618b1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r\" (UID: \"036f5ce0-df20-4e4f-b7aa-9afde4618b1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r"
Jul 15 13:58:10.762674 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.762655    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036f5ce0-df20-4e4f-b7aa-9afde4618b1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r\" (UID: \"036f5ce0-df20-4e4f-b7aa-9afde4618b1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r"
Jul 15 13:58:10.775192 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.775156    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:10.868868 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:10.868778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r"
Jul 15 13:58:11.008354 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:11.008329    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r"]
Jul 15 13:58:11.011004 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:58:11.010971    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod036f5ce0_df20_4e4f_b7aa_9afde4618b1b.slice/crio-dbf4f083ac4bb646654ce1dc811e46d5c0c07b2804bb9f5bfb13198d52680f82 WatchSource:0}: Error finding container dbf4f083ac4bb646654ce1dc811e46d5c0c07b2804bb9f5bfb13198d52680f82: Status 404 returned error can't find the container with id dbf4f083ac4bb646654ce1dc811e46d5c0c07b2804bb9f5bfb13198d52680f82
Jul 15 13:58:11.797923 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:11.797888    2577 generic.go:356] "Generic (PLEG): container finished" podID="036f5ce0-df20-4e4f-b7aa-9afde4618b1b" containerID="62da47b2fd378c32110bb8238b12217d9ded88079b951ffd858be5e42ac9c012" exitCode=0
Jul 15 13:58:11.798367 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:11.797979    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r" event={"ID":"036f5ce0-df20-4e4f-b7aa-9afde4618b1b","Type":"ContainerDied","Data":"62da47b2fd378c32110bb8238b12217d9ded88079b951ffd858be5e42ac9c012"}
Jul 15 13:58:11.798367 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:11.798044    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r" event={"ID":"036f5ce0-df20-4e4f-b7aa-9afde4618b1b","Type":"ContainerStarted","Data":"dbf4f083ac4bb646654ce1dc811e46d5c0c07b2804bb9f5bfb13198d52680f82"}
Jul 15 13:58:12.939995 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:12.939969    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r"
Jul 15 13:58:12.981367 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:12.981337    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036f5ce0-df20-4e4f-b7aa-9afde4618b1b-maas-api-service-ca\") pod \"036f5ce0-df20-4e4f-b7aa-9afde4618b1b\" (UID: \"036f5ce0-df20-4e4f-b7aa-9afde4618b1b\") "
Jul 15 13:58:12.981700 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:12.981666    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/036f5ce0-df20-4e4f-b7aa-9afde4618b1b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "036f5ce0-df20-4e4f-b7aa-9afde4618b1b" (UID: "036f5ce0-df20-4e4f-b7aa-9afde4618b1b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:58:13.082818 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:13.082735    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036f5ce0-df20-4e4f-b7aa-9afde4618b1b-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:58:13.809255 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:13.809215    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r"
Jul 15 13:58:13.809527 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:13.809290    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r" event={"ID":"036f5ce0-df20-4e4f-b7aa-9afde4618b1b","Type":"ContainerDied","Data":"dbf4f083ac4bb646654ce1dc811e46d5c0c07b2804bb9f5bfb13198d52680f82"}
Jul 15 13:58:13.809527 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:13.809352    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbf4f083ac4bb646654ce1dc811e46d5c0c07b2804bb9f5bfb13198d52680f82"
Jul 15 13:58:15.845788 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:15.845753    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"]
Jul 15 13:58:15.846198 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:15.846064    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4" podUID="99f4710d-254a-4dec-862b-0fa167a7ba39" containerName="maas-api" containerID="cri-o://df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8" gracePeriod=30
Jul 15 13:58:15.876452 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:15.876425    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:16.100796 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.100731    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:58:16.209539 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.209497    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bws69\" (UniqueName: \"kubernetes.io/projected/99f4710d-254a-4dec-862b-0fa167a7ba39-kube-api-access-bws69\") pod \"99f4710d-254a-4dec-862b-0fa167a7ba39\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") "
Jul 15 13:58:16.209725 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.209564    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99f4710d-254a-4dec-862b-0fa167a7ba39-openshift-service-ca\") pod \"99f4710d-254a-4dec-862b-0fa167a7ba39\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") "
Jul 15 13:58:16.209725 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.209609    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99f4710d-254a-4dec-862b-0fa167a7ba39-maas-api-tls\") pod \"99f4710d-254a-4dec-862b-0fa167a7ba39\" (UID: \"99f4710d-254a-4dec-862b-0fa167a7ba39\") "
Jul 15 13:58:16.209906 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.209882    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99f4710d-254a-4dec-862b-0fa167a7ba39-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99f4710d-254a-4dec-862b-0fa167a7ba39" (UID: "99f4710d-254a-4dec-862b-0fa167a7ba39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:58:16.211804 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.211776    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f4710d-254a-4dec-862b-0fa167a7ba39-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99f4710d-254a-4dec-862b-0fa167a7ba39" (UID: "99f4710d-254a-4dec-862b-0fa167a7ba39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:58:16.211912 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.211819    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f4710d-254a-4dec-862b-0fa167a7ba39-kube-api-access-bws69" (OuterVolumeSpecName: "kube-api-access-bws69") pod "99f4710d-254a-4dec-862b-0fa167a7ba39" (UID: "99f4710d-254a-4dec-862b-0fa167a7ba39"). InnerVolumeSpecName "kube-api-access-bws69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:58:16.311001 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.310963    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bws69\" (UniqueName: \"kubernetes.io/projected/99f4710d-254a-4dec-862b-0fa167a7ba39-kube-api-access-bws69\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:58:16.311001 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.310992    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99f4710d-254a-4dec-862b-0fa167a7ba39-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:58:16.311001 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.311003    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99f4710d-254a-4dec-862b-0fa167a7ba39-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:58:16.824958 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.824923    2577 generic.go:356] "Generic (PLEG): container finished" podID="99f4710d-254a-4dec-862b-0fa167a7ba39" containerID="df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8" exitCode=0
Jul 15 13:58:16.825143 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.824969    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"
Jul 15 13:58:16.825143 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.825001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4" event={"ID":"99f4710d-254a-4dec-862b-0fa167a7ba39","Type":"ContainerDied","Data":"df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8"}
Jul 15 13:58:16.825143 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.825042    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4" event={"ID":"99f4710d-254a-4dec-862b-0fa167a7ba39","Type":"ContainerDied","Data":"1118a29ad9a6463a2db7a4695d9a7d40100c35582e1a9bda067d930e072c92ae"}
Jul 15 13:58:16.825143 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.825063    2577 scope.go:117] "RemoveContainer" containerID="df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8"
Jul 15 13:58:16.833634 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.833616    2577 scope.go:117] "RemoveContainer" containerID="df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8"
Jul 15 13:58:16.833866 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:58:16.833848    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8\": container with ID starting with df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8 not found: ID does not exist" containerID="df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8"
Jul 15 13:58:16.833913 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.833878    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8"} err="failed to get container status \"df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8\": rpc error: code = NotFound desc = could not find container \"df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8\": container with ID starting with df786eab507e3445d0cf11f91253a9f4b39068f4a4e814fcb4a8dc2366e691a8 not found: ID does not exist"
Jul 15 13:58:16.861921 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.861891    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"]
Jul 15 13:58:16.870540 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:16.870516    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-65951df1-5bcd87f4b-vmgx4"]
Jul 15 13:58:17.128516 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:17.128397    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:17.240107 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:17.240077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:18.147121 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:18.147087    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99f4710d-254a-4dec-862b-0fa167a7ba39" path="/var/lib/kubelet/pods/99f4710d-254a-4dec-862b-0fa167a7ba39/volumes"
Jul 15 13:58:23.635252 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:23.635215    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:23.760423 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:23.760392    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:23.942507 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:23.942422    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:25.037473 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.037412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:25.158877 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.158849    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:25.293225 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.293151    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"]
Jul 15 13:58:25.293511 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.293497    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerName="revoke-keys"
Jul 15 13:58:25.293555 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.293512    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerName="revoke-keys"
Jul 15 13:58:25.293555 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.293524    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f4710d-254a-4dec-862b-0fa167a7ba39" containerName="maas-api"
Jul 15 13:58:25.293555 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.293530    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f4710d-254a-4dec-862b-0fa167a7ba39" containerName="maas-api"
Jul 15 13:58:25.293555 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.293538    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="036f5ce0-df20-4e4f-b7aa-9afde4618b1b" containerName="revoke-keys"
Jul 15 13:58:25.293555 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.293543    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="036f5ce0-df20-4e4f-b7aa-9afde4618b1b" containerName="revoke-keys"
Jul 15 13:58:25.293708 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.293604    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdefd2ee-5b80-422b-ac17-1214dafbaf7f" containerName="revoke-keys"
Jul 15 13:58:25.293708 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.293612    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f4710d-254a-4dec-862b-0fa167a7ba39" containerName="maas-api"
Jul 15 13:58:25.293708 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.293618    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="036f5ce0-df20-4e4f-b7aa-9afde4618b1b" containerName="revoke-keys"
Jul 15 13:58:25.297778 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.297757    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:25.303214 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.303195    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0e9155d7-serving-cert\""
Jul 15 13:58:25.319542 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.319510    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"]
Jul 15 13:58:25.347866 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.347837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:25.388668 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.388631    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c5b829e-13dc-4647-bce8-0254b021a2f6-openshift-service-ca\") pod \"maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:25.388837 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.388790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv6vx\" (UniqueName: \"kubernetes.io/projected/5c5b829e-13dc-4647-bce8-0254b021a2f6-kube-api-access-fv6vx\") pod \"maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:25.388837 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.388831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c5b829e-13dc-4647-bce8-0254b021a2f6-maas-api-tls\") pod \"maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:25.489450 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.489418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c5b829e-13dc-4647-bce8-0254b021a2f6-maas-api-tls\") pod \"maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:25.489630 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.489469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c5b829e-13dc-4647-bce8-0254b021a2f6-openshift-service-ca\") pod \"maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:25.489630 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.489513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv6vx\" (UniqueName: \"kubernetes.io/projected/5c5b829e-13dc-4647-bce8-0254b021a2f6-kube-api-access-fv6vx\") pod \"maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:25.489630 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:58:25.489589    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-serving-cert: secret "maas-api-e2e-ait-0e9155d7-serving-cert" not found
Jul 15 13:58:25.489738 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:58:25.489672    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c5b829e-13dc-4647-bce8-0254b021a2f6-maas-api-tls podName:5c5b829e-13dc-4647-bce8-0254b021a2f6 nodeName:}" failed. No retries permitted until 2026-07-15 13:58:25.989652346 +0000 UTC m=+1750.332587424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5c5b829e-13dc-4647-bce8-0254b021a2f6-maas-api-tls") pod "maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v" (UID: "5c5b829e-13dc-4647-bce8-0254b021a2f6") : secret "maas-api-e2e-ait-0e9155d7-serving-cert" not found
Jul 15 13:58:25.490089 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.490072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c5b829e-13dc-4647-bce8-0254b021a2f6-openshift-service-ca\") pod \"maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:25.519403 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.519375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv6vx\" (UniqueName: \"kubernetes.io/projected/5c5b829e-13dc-4647-bce8-0254b021a2f6-kube-api-access-fv6vx\") pod \"maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:25.769973 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.769936    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:25.993876 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.993835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c5b829e-13dc-4647-bce8-0254b021a2f6-maas-api-tls\") pod \"maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:25.996577 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:25.996554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c5b829e-13dc-4647-bce8-0254b021a2f6-maas-api-tls\") pod \"maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:26.208107 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:26.208023    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:26.212304 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:26.212223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:26.333591 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:26.333565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:26.391033 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:26.390895    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"]
Jul 15 13:58:26.393765 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:58:26.393728    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c5b829e_13dc_4647_bce8_0254b021a2f6.slice/crio-61718e87e1d260fe1f05f5473f0a25f7a45635f4aeae28a4dbba721c664654b1 WatchSource:0}: Error finding container 61718e87e1d260fe1f05f5473f0a25f7a45635f4aeae28a4dbba721c664654b1: Status 404 returned error can't find the container with id 61718e87e1d260fe1f05f5473f0a25f7a45635f4aeae28a4dbba721c664654b1
Jul 15 13:58:26.874880 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:26.874828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v" event={"ID":"5c5b829e-13dc-4647-bce8-0254b021a2f6","Type":"ContainerStarted","Data":"61718e87e1d260fe1f05f5473f0a25f7a45635f4aeae28a4dbba721c664654b1"}
Jul 15 13:58:27.002196 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:27.002157    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:27.880989 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:27.880955    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v" event={"ID":"5c5b829e-13dc-4647-bce8-0254b021a2f6","Type":"ContainerStarted","Data":"b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd"}
Jul 15 13:58:27.881471 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:27.881076    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:27.913636 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:27.913577    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v" podStartSLOduration=2.457006874 podStartE2EDuration="2.913555667s" totalImagesPullingTime="456.548793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:58:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:58:27.911041708 +0000 UTC m=+1752.253976787" watchObservedRunningTime="2026-07-15 13:58:27.913555667 +0000 UTC m=+1752.256490748"
Jul 15 13:58:30.158120 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:30.158083    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"]
Jul 15 13:58:30.161469 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:30.161452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"
Jul 15 13:58:30.168053 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:30.168031    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 13:58:30.224797 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:30.224758    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"]
Jul 15 13:58:30.236943 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:30.236901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac4afc80-ee3b-40be-a833-2217d7c88aa1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9\" (UID: \"ac4afc80-ee3b-40be-a833-2217d7c88aa1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"
Jul 15 13:58:30.337933 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:30.337898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac4afc80-ee3b-40be-a833-2217d7c88aa1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9\" (UID: \"ac4afc80-ee3b-40be-a833-2217d7c88aa1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"
Jul 15 13:58:30.338552 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:30.338533    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac4afc80-ee3b-40be-a833-2217d7c88aa1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9\" (UID: \"ac4afc80-ee3b-40be-a833-2217d7c88aa1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"
Jul 15 13:58:30.471118 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:30.471044    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"
Jul 15 13:58:30.625842 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:30.625810    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"]
Jul 15 13:58:30.628741 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:58:30.628705    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4afc80_ee3b_40be_a833_2217d7c88aa1.slice/crio-894bd1cb0943b0495eba9653cbba9cd2151691bcdf7b0004a7d317a8f72807aa WatchSource:0}: Error finding container 894bd1cb0943b0495eba9653cbba9cd2151691bcdf7b0004a7d317a8f72807aa: Status 404 returned error can't find the container with id 894bd1cb0943b0495eba9653cbba9cd2151691bcdf7b0004a7d317a8f72807aa
Jul 15 13:58:30.896561 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:30.896530    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9" event={"ID":"ac4afc80-ee3b-40be-a833-2217d7c88aa1","Type":"ContainerStarted","Data":"894bd1cb0943b0495eba9653cbba9cd2151691bcdf7b0004a7d317a8f72807aa"}
Jul 15 13:58:31.907004 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:31.906968    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerID="9f328668ce1e1532d3431a2f1c56b32e86c23eae33ae48bf16bd75413c052cd3" exitCode=7
Jul 15 13:58:31.907472 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:31.907053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9" event={"ID":"ac4afc80-ee3b-40be-a833-2217d7c88aa1","Type":"ContainerDied","Data":"9f328668ce1e1532d3431a2f1c56b32e86c23eae33ae48bf16bd75413c052cd3"}
Jul 15 13:58:31.907472 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:31.907234    2577 scope.go:117] "RemoveContainer" containerID="9f328668ce1e1532d3431a2f1c56b32e86c23eae33ae48bf16bd75413c052cd3"
Jul 15 13:58:32.913519 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:32.913481    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerID="b6e6c72e12c2856b0dae388b43c715324b24d9d146cf9de3f55fd8b0ae963061" exitCode=7
Jul 15 13:58:32.913920 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:32.913567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9" event={"ID":"ac4afc80-ee3b-40be-a833-2217d7c88aa1","Type":"ContainerDied","Data":"b6e6c72e12c2856b0dae388b43c715324b24d9d146cf9de3f55fd8b0ae963061"}
Jul 15 13:58:32.913920 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:32.913620    2577 scope.go:117] "RemoveContainer" containerID="9f328668ce1e1532d3431a2f1c56b32e86c23eae33ae48bf16bd75413c052cd3"
Jul 15 13:58:32.913920 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:32.913818    2577 scope.go:117] "RemoveContainer" containerID="b6e6c72e12c2856b0dae388b43c715324b24d9d146cf9de3f55fd8b0ae963061"
Jul 15 13:58:32.914062 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:58:32.914032    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9_odh-ai-gateway-infra(ac4afc80-ee3b-40be-a833-2217d7c88aa1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1"
Jul 15 13:58:33.890834 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:33.890799    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:58:33.919941 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:33.919913    2577 scope.go:117] "RemoveContainer" containerID="b6e6c72e12c2856b0dae388b43c715324b24d9d146cf9de3f55fd8b0ae963061"
Jul 15 13:58:33.920341 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:58:33.920102    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9_odh-ai-gateway-infra(ac4afc80-ee3b-40be-a833-2217d7c88aa1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1"
Jul 15 13:58:34.665390 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:34.665353    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:34.767162 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:34.767129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:47.141141 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:47.141111    2577 scope.go:117] "RemoveContainer" containerID="b6e6c72e12c2856b0dae388b43c715324b24d9d146cf9de3f55fd8b0ae963061"
Jul 15 13:58:47.239758 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:47.239726    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:47.335724 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:47.335689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:58:47.986363 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:47.986332    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerID="1e5e8f54a14f40069cba43667085ecd4a041caf707f86f727f1a5f4b002a2870" exitCode=0
Jul 15 13:58:47.986548 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:47.986415    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9" event={"ID":"ac4afc80-ee3b-40be-a833-2217d7c88aa1","Type":"ContainerDied","Data":"1e5e8f54a14f40069cba43667085ecd4a041caf707f86f727f1a5f4b002a2870"}
Jul 15 13:58:47.986548 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:47.986482    2577 scope.go:117] "RemoveContainer" containerID="b6e6c72e12c2856b0dae388b43c715324b24d9d146cf9de3f55fd8b0ae963061"
Jul 15 13:58:49.019975 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:49.019944    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"]
Jul 15 13:58:49.132246 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:49.132219    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"
Jul 15 13:58:49.307308 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:49.307256    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac4afc80-ee3b-40be-a833-2217d7c88aa1-maas-api-service-ca\") pod \"ac4afc80-ee3b-40be-a833-2217d7c88aa1\" (UID: \"ac4afc80-ee3b-40be-a833-2217d7c88aa1\") "
Jul 15 13:58:49.307608 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:49.307587    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac4afc80-ee3b-40be-a833-2217d7c88aa1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ac4afc80-ee3b-40be-a833-2217d7c88aa1" (UID: "ac4afc80-ee3b-40be-a833-2217d7c88aa1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:58:49.408581 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:49.408546    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac4afc80-ee3b-40be-a833-2217d7c88aa1-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:58:50.000004 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:49.999970    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9" event={"ID":"ac4afc80-ee3b-40be-a833-2217d7c88aa1","Type":"ContainerDied","Data":"894bd1cb0943b0495eba9653cbba9cd2151691bcdf7b0004a7d317a8f72807aa"}
Jul 15 13:58:50.000171 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:50.000018    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="894bd1cb0943b0495eba9653cbba9cd2151691bcdf7b0004a7d317a8f72807aa"
Jul 15 13:58:50.000171 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:49.999979    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"
Jul 15 13:58:50.032239 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:50.032206    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"]
Jul 15 13:58:50.041180 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:50.041155    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-bmpc9"]
Jul 15 13:58:50.148296 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:58:50.148241    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" path="/var/lib/kubelet/pods/ac4afc80-ee3b-40be-a833-2217d7c88aa1/volumes"
Jul 15 13:59:16.171803 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:16.171769    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:59:16.175637 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:16.175611    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 13:59:20.194019 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.193983    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf"]
Jul 15 13:59:20.194499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.194347    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerName="revoke-keys"
Jul 15 13:59:20.194499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.194360    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerName="revoke-keys"
Jul 15 13:59:20.194499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.194375    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerName="revoke-keys"
Jul 15 13:59:20.194499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.194381    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerName="revoke-keys"
Jul 15 13:59:20.194499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.194393    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerName="revoke-keys"
Jul 15 13:59:20.194499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.194399    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerName="revoke-keys"
Jul 15 13:59:20.194499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.194448    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerName="revoke-keys"
Jul 15 13:59:20.194499 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.194456    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerName="revoke-keys"
Jul 15 13:59:20.197727 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.197711    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf"
Jul 15 13:59:20.200460 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.200438    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 13:59:20.205371 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.205338    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf"]
Jul 15 13:59:20.267217 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.267179    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf\" (UID: \"ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf"
Jul 15 13:59:20.368522 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.368483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf\" (UID: \"ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf"
Jul 15 13:59:20.369110 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.369090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf\" (UID: \"ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf"
Jul 15 13:59:20.510752 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.510719    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf"
Jul 15 13:59:20.650006 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:20.649969    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf"]
Jul 15 13:59:20.652821 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:59:20.652791    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae91d8cb_ede2_48fa_90e8_9dc2e9b4277c.slice/crio-5b857f20bdba3140f1f821a73817d3b395e49b870b8c9d6120212bc92a711885 WatchSource:0}: Error finding container 5b857f20bdba3140f1f821a73817d3b395e49b870b8c9d6120212bc92a711885: Status 404 returned error can't find the container with id 5b857f20bdba3140f1f821a73817d3b395e49b870b8c9d6120212bc92a711885
Jul 15 13:59:21.152729 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:21.152628    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf" event={"ID":"ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c","Type":"ContainerStarted","Data":"ae74cff8753b152e34abd0ff9ef3fa99a0c26a6f5d58822de737251b65bdc952"}
Jul 15 13:59:21.152729 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:21.152696    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf" event={"ID":"ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c","Type":"ContainerStarted","Data":"5b857f20bdba3140f1f821a73817d3b395e49b870b8c9d6120212bc92a711885"}
Jul 15 13:59:21.171202 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:21.171140    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf" podStartSLOduration=1.171122623 podStartE2EDuration="1.171122623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:21.169446857 +0000 UTC m=+1805.512381957" watchObservedRunningTime="2026-07-15 13:59:21.171122623 +0000 UTC m=+1805.514057702"
Jul 15 13:59:22.158316 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:22.158283    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c" containerID="ae74cff8753b152e34abd0ff9ef3fa99a0c26a6f5d58822de737251b65bdc952" exitCode=0
Jul 15 13:59:22.158697 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:22.158322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf" event={"ID":"ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c","Type":"ContainerDied","Data":"ae74cff8753b152e34abd0ff9ef3fa99a0c26a6f5d58822de737251b65bdc952"}
Jul 15 13:59:23.304239 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:23.304207    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf"
Jul 15 13:59:23.397150 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:23.397119    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c-maas-api-service-ca\") pod \"ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c\" (UID: \"ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c\") "
Jul 15 13:59:23.397501 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:23.397479    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c" (UID: "ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:59:23.498395 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:23.498306    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:59:24.169805 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:24.169774    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf" event={"ID":"ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c","Type":"ContainerDied","Data":"5b857f20bdba3140f1f821a73817d3b395e49b870b8c9d6120212bc92a711885"}
Jul 15 13:59:24.169953 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:24.169811    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf"
Jul 15 13:59:24.169953 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:24.169822    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b857f20bdba3140f1f821a73817d3b395e49b870b8c9d6120212bc92a711885"
Jul 15 13:59:30.841755 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:30.841719    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"]
Jul 15 13:59:30.842211 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:30.841998    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v" podUID="5c5b829e-13dc-4647-bce8-0254b021a2f6" containerName="maas-api" containerID="cri-o://b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd" gracePeriod=30
Jul 15 13:59:30.860534 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:30.860505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:31.115535 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.115511    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:59:31.202573 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.202535    2577 generic.go:356] "Generic (PLEG): container finished" podID="5c5b829e-13dc-4647-bce8-0254b021a2f6" containerID="b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd" exitCode=0
Jul 15 13:59:31.202759 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.202579    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"
Jul 15 13:59:31.202759 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.202622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v" event={"ID":"5c5b829e-13dc-4647-bce8-0254b021a2f6","Type":"ContainerDied","Data":"b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd"}
Jul 15 13:59:31.202759 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.202680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v" event={"ID":"5c5b829e-13dc-4647-bce8-0254b021a2f6","Type":"ContainerDied","Data":"61718e87e1d260fe1f05f5473f0a25f7a45635f4aeae28a4dbba721c664654b1"}
Jul 15 13:59:31.202759 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.202707    2577 scope.go:117] "RemoveContainer" containerID="b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd"
Jul 15 13:59:31.211254 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.211234    2577 scope.go:117] "RemoveContainer" containerID="b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd"
Jul 15 13:59:31.211563 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:59:31.211539    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd\": container with ID starting with b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd not found: ID does not exist" containerID="b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd"
Jul 15 13:59:31.211662 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.211572    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd"} err="failed to get container status \"b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd\": rpc error: code = NotFound desc = could not find container \"b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd\": container with ID starting with b0731872aa2268197ff779bb4d70adc23a81c363443605e0cd4b24e37ceec7dd not found: ID does not exist"
Jul 15 13:59:31.268080 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.268045    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fv6vx\" (UniqueName: \"kubernetes.io/projected/5c5b829e-13dc-4647-bce8-0254b021a2f6-kube-api-access-fv6vx\") pod \"5c5b829e-13dc-4647-bce8-0254b021a2f6\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") "
Jul 15 13:59:31.268236 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.268112    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c5b829e-13dc-4647-bce8-0254b021a2f6-openshift-service-ca\") pod \"5c5b829e-13dc-4647-bce8-0254b021a2f6\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") "
Jul 15 13:59:31.268236 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.268186    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c5b829e-13dc-4647-bce8-0254b021a2f6-maas-api-tls\") pod \"5c5b829e-13dc-4647-bce8-0254b021a2f6\" (UID: \"5c5b829e-13dc-4647-bce8-0254b021a2f6\") "
Jul 15 13:59:31.268472 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.268443    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5b829e-13dc-4647-bce8-0254b021a2f6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5c5b829e-13dc-4647-bce8-0254b021a2f6" (UID: "5c5b829e-13dc-4647-bce8-0254b021a2f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:59:31.270403 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.270365    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5b829e-13dc-4647-bce8-0254b021a2f6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5c5b829e-13dc-4647-bce8-0254b021a2f6" (UID: "5c5b829e-13dc-4647-bce8-0254b021a2f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:59:31.270526 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.270411    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c5b829e-13dc-4647-bce8-0254b021a2f6-kube-api-access-fv6vx" (OuterVolumeSpecName: "kube-api-access-fv6vx") pod "5c5b829e-13dc-4647-bce8-0254b021a2f6" (UID: "5c5b829e-13dc-4647-bce8-0254b021a2f6"). InnerVolumeSpecName "kube-api-access-fv6vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:59:31.369533 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.369450    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c5b829e-13dc-4647-bce8-0254b021a2f6-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:59:31.369533 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.369482    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv6vx\" (UniqueName: \"kubernetes.io/projected/5c5b829e-13dc-4647-bce8-0254b021a2f6-kube-api-access-fv6vx\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:59:31.369533 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.369491    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c5b829e-13dc-4647-bce8-0254b021a2f6-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 13:59:31.528168 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.528137    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"]
Jul 15 13:59:31.531536 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:31.531510    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e9155d7-7f747c8f64-2k44v"]
Jul 15 13:59:32.147196 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:32.147150    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c5b829e-13dc-4647-bce8-0254b021a2f6" path="/var/lib/kubelet/pods/5c5b829e-13dc-4647-bce8-0254b021a2f6/volumes"
Jul 15 13:59:35.275549 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:35.275514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:41.574225 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:41.574187    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:41.650649 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:41.650615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:46.395494 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:46.395461    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:46.735604 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:46.735567    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:47.886135 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.886096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:47.956140 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.956101    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"]
Jul 15 13:59:47.956675 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.956652    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c" containerName="revoke-keys"
Jul 15 13:59:47.956675 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.956674    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c" containerName="revoke-keys"
Jul 15 13:59:47.956839 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.956702    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5b829e-13dc-4647-bce8-0254b021a2f6" containerName="maas-api"
Jul 15 13:59:47.956839 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.956711    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5b829e-13dc-4647-bce8-0254b021a2f6" containerName="maas-api"
Jul 15 13:59:47.956839 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.956808    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c" containerName="revoke-keys"
Jul 15 13:59:47.956839 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.956825    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac4afc80-ee3b-40be-a833-2217d7c88aa1" containerName="revoke-keys"
Jul 15 13:59:47.956839 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.956835    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c5b829e-13dc-4647-bce8-0254b021a2f6" containerName="maas-api"
Jul 15 13:59:47.959891 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.959870    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:47.963805 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.963786    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6ff521e6-serving-cert\""
Jul 15 13:59:47.977673 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.977638    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"]
Jul 15 13:59:47.988593 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:47.988560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:48.011799 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.011767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnwjf\" (UniqueName: \"kubernetes.io/projected/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-kube-api-access-qnwjf\") pod \"maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:48.011964 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.011845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-maas-api-tls\") pod \"maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:48.011964 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.011892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-openshift-service-ca\") pod \"maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:48.076036 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.076002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:48.112350 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.112315    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnwjf\" (UniqueName: \"kubernetes.io/projected/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-kube-api-access-qnwjf\") pod \"maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:48.112546 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.112378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-maas-api-tls\") pod \"maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:48.112546 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.112418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-openshift-service-ca\") pod \"maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:48.112671 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:59:48.112556    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-serving-cert: secret "maas-api-e2e-derive-6ff521e6-serving-cert" not found
Jul 15 13:59:48.112671 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:59:48.112629    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-maas-api-tls podName:67684a2d-b3d7-4e74-a4ea-a8a7bb421196 nodeName:}" failed. No retries permitted until 2026-07-15 13:59:48.612606104 +0000 UTC m=+1832.955541178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-maas-api-tls") pod "maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq" (UID: "67684a2d-b3d7-4e74-a4ea-a8a7bb421196") : secret "maas-api-e2e-derive-6ff521e6-serving-cert" not found
Jul 15 13:59:48.113128 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.113109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-openshift-service-ca\") pod \"maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:48.127131 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.127102    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnwjf\" (UniqueName: \"kubernetes.io/projected/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-kube-api-access-qnwjf\") pod \"maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:48.249188 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.249073    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:48.598475 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.598445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:48.617596 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.617564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-maas-api-tls\") pod \"maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:48.620178 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.620147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-maas-api-tls\") pod \"maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:48.704476 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.704448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 13:59:48.871145 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:48.871050    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:49.029160 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:49.029121    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"]
Jul 15 13:59:49.033149 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:59:49.033121    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67684a2d_b3d7_4e74_a4ea_a8a7bb421196.slice/crio-a9b4bd2dc98d56f8ac3a087ec6af97b9ed17c02fa48a70ee770fd11631b68cdd WatchSource:0}: Error finding container a9b4bd2dc98d56f8ac3a087ec6af97b9ed17c02fa48a70ee770fd11631b68cdd: Status 404 returned error can't find the container with id a9b4bd2dc98d56f8ac3a087ec6af97b9ed17c02fa48a70ee770fd11631b68cdd
Jul 15 13:59:49.292175 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:49.292141    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq" event={"ID":"67684a2d-b3d7-4e74-a4ea-a8a7bb421196","Type":"ContainerStarted","Data":"a9b4bd2dc98d56f8ac3a087ec6af97b9ed17c02fa48a70ee770fd11631b68cdd"}
Jul 15 13:59:51.303954 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:51.303915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq" event={"ID":"67684a2d-b3d7-4e74-a4ea-a8a7bb421196","Type":"ContainerStarted","Data":"ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75"}
Jul 15 13:59:51.304513 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:51.303993    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 13:59:51.330047 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:51.329999    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq" podStartSLOduration=2.916996608 podStartE2EDuration="4.329983864s" totalImagesPullingTime="1.412987256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:59:51.328693669 +0000 UTC m=+1835.671628764" watchObservedRunningTime="2026-07-15 13:59:51.329983864 +0000 UTC m=+1835.672918944"
Jul 15 13:59:52.264692 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:52.264655    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"]
Jul 15 13:59:52.268199 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:52.268180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"
Jul 15 13:59:52.270853 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:52.270831    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 13:59:52.277414 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:52.277393    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"]
Jul 15 13:59:52.352432 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:52.352396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e06e2c5-7957-48ed-88f5-36b73e5f452e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz\" (UID: \"4e06e2c5-7957-48ed-88f5-36b73e5f452e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"
Jul 15 13:59:52.453485 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:52.453446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e06e2c5-7957-48ed-88f5-36b73e5f452e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz\" (UID: \"4e06e2c5-7957-48ed-88f5-36b73e5f452e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"
Jul 15 13:59:52.454135 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:52.454108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e06e2c5-7957-48ed-88f5-36b73e5f452e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz\" (UID: \"4e06e2c5-7957-48ed-88f5-36b73e5f452e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"
Jul 15 13:59:52.580009 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:52.579910    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"
Jul 15 13:59:52.724679 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:52.724652    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"]
Jul 15 13:59:52.727629 ip-10-0-141-134 kubenswrapper[2577]: W0715 13:59:52.727587    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e06e2c5_7957_48ed_88f5_36b73e5f452e.slice/crio-5b148c5e643882e14929f6e593a8d329ffa8b1d43b334ec89aabea5554eb2500 WatchSource:0}: Error finding container 5b148c5e643882e14929f6e593a8d329ffa8b1d43b334ec89aabea5554eb2500: Status 404 returned error can't find the container with id 5b148c5e643882e14929f6e593a8d329ffa8b1d43b334ec89aabea5554eb2500
Jul 15 13:59:53.315884 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:53.315848    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerID="63ef27a9363ab6d9132673c51e92a7786427f1ec986cd660e90bfdc15a1fcc0a" exitCode=7
Jul 15 13:59:53.316055 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:53.315929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz" event={"ID":"4e06e2c5-7957-48ed-88f5-36b73e5f452e","Type":"ContainerDied","Data":"63ef27a9363ab6d9132673c51e92a7786427f1ec986cd660e90bfdc15a1fcc0a"}
Jul 15 13:59:53.316055 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:53.316013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz" event={"ID":"4e06e2c5-7957-48ed-88f5-36b73e5f452e","Type":"ContainerStarted","Data":"5b148c5e643882e14929f6e593a8d329ffa8b1d43b334ec89aabea5554eb2500"}
Jul 15 13:59:53.316162 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:53.316143    2577 scope.go:117] "RemoveContainer" containerID="63ef27a9363ab6d9132673c51e92a7786427f1ec986cd660e90bfdc15a1fcc0a"
Jul 15 13:59:54.323405 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:54.323373    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerID="c3f273a24657a1f575c42aa888172dfc2601b691377387fee6b15324a75e15a0" exitCode=7
Jul 15 13:59:54.323810 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:54.323460    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz" event={"ID":"4e06e2c5-7957-48ed-88f5-36b73e5f452e","Type":"ContainerDied","Data":"c3f273a24657a1f575c42aa888172dfc2601b691377387fee6b15324a75e15a0"}
Jul 15 13:59:54.323810 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:54.323521    2577 scope.go:117] "RemoveContainer" containerID="63ef27a9363ab6d9132673c51e92a7786427f1ec986cd660e90bfdc15a1fcc0a"
Jul 15 13:59:54.323810 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:54.323708    2577 scope.go:117] "RemoveContainer" containerID="c3f273a24657a1f575c42aa888172dfc2601b691377387fee6b15324a75e15a0"
Jul 15 13:59:54.323933 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:59:54.323909    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz_odh-ai-gateway-infra(4e06e2c5-7957-48ed-88f5-36b73e5f452e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e"
Jul 15 13:59:55.329512 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:55.329485    2577 scope.go:117] "RemoveContainer" containerID="c3f273a24657a1f575c42aa888172dfc2601b691377387fee6b15324a75e15a0"
Jul 15 13:59:55.329894 ip-10-0-141-134 kubenswrapper[2577]: E0715 13:59:55.329676    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz_odh-ai-gateway-infra(4e06e2c5-7957-48ed-88f5-36b73e5f452e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e"
Jul 15 13:59:57.314916 ip-10-0-141-134 kubenswrapper[2577]: I0715 13:59:57.314883    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 14:00:00.147325 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:00.147297    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh"]
Jul 15 14:00:00.151957 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:00.151937    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p"]
Jul 15 14:00:00.152066 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:00.152046    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh"
Jul 15 14:00:00.155208 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:00.155186    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p"
Jul 15 14:00:00.161637 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:00.161595    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh"]
Jul 15 14:00:00.163975 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:00.163953    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p"]
Jul 15 14:00:00.164716 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:00.164693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh"
Jul 15 14:00:00.172193 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:00.172174    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p"
Jul 15 14:00:00.535799 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:00.535771    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh"]
Jul 15 14:00:00.538424 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:00:00.538393    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c2521c_b6cb_4cbd_a842_99e61e7c963a.slice/crio-0b1f9420bc9cf5c246f7c487db4ede53925cae6bf498b169a8e85acdce986289 WatchSource:0}: Error finding container 0b1f9420bc9cf5c246f7c487db4ede53925cae6bf498b169a8e85acdce986289: Status 404 returned error can't find the container with id 0b1f9420bc9cf5c246f7c487db4ede53925cae6bf498b169a8e85acdce986289
Jul 15 14:00:00.559989 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:00.559950    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p"]
Jul 15 14:00:00.561839 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:00:00.561806    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e6d1e4_51c2_478e_af8c_f0095c746405.slice/crio-aa084253c84d9bafb88e202a4adbd197c0a6562f5d3c90bbe79f7241d36581fe WatchSource:0}: Error finding container aa084253c84d9bafb88e202a4adbd197c0a6562f5d3c90bbe79f7241d36581fe: Status 404 returned error can't find the container with id aa084253c84d9bafb88e202a4adbd197c0a6562f5d3c90bbe79f7241d36581fe
Jul 15 14:00:01.358048 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:01.358008    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh" event={"ID":"33c2521c-b6cb-4cbd-a842-99e61e7c963a","Type":"ContainerStarted","Data":"6190ca2432e0e00292a873d02f8fd4a0c7e0658aab34272eb35d951c10431606"}
Jul 15 14:00:01.358523 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:01.358086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh" event={"ID":"33c2521c-b6cb-4cbd-a842-99e61e7c963a","Type":"ContainerStarted","Data":"0b1f9420bc9cf5c246f7c487db4ede53925cae6bf498b169a8e85acdce986289"}
Jul 15 14:00:01.359587 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:01.359561    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p" event={"ID":"92e6d1e4-51c2-478e-af8c-f0095c746405","Type":"ContainerStarted","Data":"6c62fa95a79d8484581bc093cd70ddea6f712fddd8b1391db0f983fe90a1eca1"}
Jul 15 14:00:01.359710 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:01.359606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p" event={"ID":"92e6d1e4-51c2-478e-af8c-f0095c746405","Type":"ContainerStarted","Data":"aa084253c84d9bafb88e202a4adbd197c0a6562f5d3c90bbe79f7241d36581fe"}
Jul 15 14:00:01.378237 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:01.378178    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh" podStartSLOduration=1.378161116 podStartE2EDuration="1.378161116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:00:01.37624549 +0000 UTC m=+1845.719180570" watchObservedRunningTime="2026-07-15 14:00:01.378161116 +0000 UTC m=+1845.721096225"
Jul 15 14:00:01.398016 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:01.397951    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p" podStartSLOduration=1.397928661 podStartE2EDuration="1.397928661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:00:01.39668976 +0000 UTC m=+1845.739624840" watchObservedRunningTime="2026-07-15 14:00:01.397928661 +0000 UTC m=+1845.740863742"
Jul 15 14:00:02.365804 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:02.365769    2577 generic.go:356] "Generic (PLEG): container finished" podID="33c2521c-b6cb-4cbd-a842-99e61e7c963a" containerID="6190ca2432e0e00292a873d02f8fd4a0c7e0658aab34272eb35d951c10431606" exitCode=0
Jul 15 14:00:02.366284 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:02.365852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh" event={"ID":"33c2521c-b6cb-4cbd-a842-99e61e7c963a","Type":"ContainerDied","Data":"6190ca2432e0e00292a873d02f8fd4a0c7e0658aab34272eb35d951c10431606"}
Jul 15 14:00:02.367327 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:02.367299    2577 generic.go:356] "Generic (PLEG): container finished" podID="92e6d1e4-51c2-478e-af8c-f0095c746405" containerID="6c62fa95a79d8484581bc093cd70ddea6f712fddd8b1391db0f983fe90a1eca1" exitCode=0
Jul 15 14:00:02.367450 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:02.367343    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p" event={"ID":"92e6d1e4-51c2-478e-af8c-f0095c746405","Type":"ContainerDied","Data":"6c62fa95a79d8484581bc093cd70ddea6f712fddd8b1391db0f983fe90a1eca1"}
Jul 15 14:00:03.513539 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:03.513516    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p"
Jul 15 14:00:03.550477 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:03.550453    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh"
Jul 15 14:00:03.975830 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:03.975801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:00:04.378654 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:04.378623    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh"
Jul 15 14:00:04.378856 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:04.378628    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735400-tb4mh" event={"ID":"33c2521c-b6cb-4cbd-a842-99e61e7c963a","Type":"ContainerDied","Data":"0b1f9420bc9cf5c246f7c487db4ede53925cae6bf498b169a8e85acdce986289"}
Jul 15 14:00:04.378856 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:04.378779    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b1f9420bc9cf5c246f7c487db4ede53925cae6bf498b169a8e85acdce986289"
Jul 15 14:00:04.379867 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:04.379845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p" event={"ID":"92e6d1e4-51c2-478e-af8c-f0095c746405","Type":"ContainerDied","Data":"aa084253c84d9bafb88e202a4adbd197c0a6562f5d3c90bbe79f7241d36581fe"}
Jul 15 14:00:04.379993 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:04.379883    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa084253c84d9bafb88e202a4adbd197c0a6562f5d3c90bbe79f7241d36581fe"
Jul 15 14:00:04.379993 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:04.379883    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p"
Jul 15 14:00:05.764213 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:05.764177    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:00:08.129342 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:08.129313    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:00:09.141531 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:09.141503    2577 scope.go:117] "RemoveContainer" containerID="c3f273a24657a1f575c42aa888172dfc2601b691377387fee6b15324a75e15a0"
Jul 15 14:00:09.405358 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:09.405255    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz" event={"ID":"4e06e2c5-7957-48ed-88f5-36b73e5f452e","Type":"ContainerStarted","Data":"b609a720e17ad87c425604630420a0eae94d44e2d080d38c1166f6fea76581e6"}
Jul 15 14:00:09.426344 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:09.426296    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz" podStartSLOduration=17.426280877 podStartE2EDuration="17.426280877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 13:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:00:09.42532239 +0000 UTC m=+1853.768257496" watchObservedRunningTime="2026-07-15 14:00:09.426280877 +0000 UTC m=+1853.769215947"
Jul 15 14:00:10.411632 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:10.411595    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerID="b609a720e17ad87c425604630420a0eae94d44e2d080d38c1166f6fea76581e6" exitCode=0
Jul 15 14:00:10.412023 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:10.411664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz" event={"ID":"4e06e2c5-7957-48ed-88f5-36b73e5f452e","Type":"ContainerDied","Data":"b609a720e17ad87c425604630420a0eae94d44e2d080d38c1166f6fea76581e6"}
Jul 15 14:00:10.412023 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:10.411728    2577 scope.go:117] "RemoveContainer" containerID="c3f273a24657a1f575c42aa888172dfc2601b691377387fee6b15324a75e15a0"
Jul 15 14:00:10.441522 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:10.441491    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"]
Jul 15 14:00:11.552093 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:11.552068    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"
Jul 15 14:00:11.623792 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:11.623755    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e06e2c5-7957-48ed-88f5-36b73e5f452e-maas-api-service-ca\") pod \"4e06e2c5-7957-48ed-88f5-36b73e5f452e\" (UID: \"4e06e2c5-7957-48ed-88f5-36b73e5f452e\") "
Jul 15 14:00:11.624131 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:11.624105    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e06e2c5-7957-48ed-88f5-36b73e5f452e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e06e2c5-7957-48ed-88f5-36b73e5f452e" (UID: "4e06e2c5-7957-48ed-88f5-36b73e5f452e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:00:11.724640 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:11.724553    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e06e2c5-7957-48ed-88f5-36b73e5f452e-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:00:12.423841 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:12.423759    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"
Jul 15 14:00:12.423841 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:12.423772    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz" event={"ID":"4e06e2c5-7957-48ed-88f5-36b73e5f452e","Type":"ContainerDied","Data":"5b148c5e643882e14929f6e593a8d329ffa8b1d43b334ec89aabea5554eb2500"}
Jul 15 14:00:12.423841 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:12.423816    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b148c5e643882e14929f6e593a8d329ffa8b1d43b334ec89aabea5554eb2500"
Jul 15 14:00:12.479688 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:12.479651    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"]
Jul 15 14:00:12.486287 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:12.486251    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-sbfzz"]
Jul 15 14:00:14.146697 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:14.146662    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" path="/var/lib/kubelet/pods/4e06e2c5-7957-48ed-88f5-36b73e5f452e/volumes"
Jul 15 14:00:52.348978 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.348934    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b"]
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349330    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerName="revoke-keys"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349342    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerName="revoke-keys"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349354    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33c2521c-b6cb-4cbd-a842-99e61e7c963a" containerName="cleanup"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349359    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="33c2521c-b6cb-4cbd-a842-99e61e7c963a" containerName="cleanup"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349369    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92e6d1e4-51c2-478e-af8c-f0095c746405" containerName="cleanup"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349375    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="92e6d1e4-51c2-478e-af8c-f0095c746405" containerName="cleanup"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349385    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerName="revoke-keys"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349390    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerName="revoke-keys"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349442    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerName="revoke-keys"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349448    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="33c2521c-b6cb-4cbd-a842-99e61e7c963a" containerName="cleanup"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349454    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerName="revoke-keys"
Jul 15 14:00:52.349547 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.349459    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="92e6d1e4-51c2-478e-af8c-f0095c746405" containerName="cleanup"
Jul 15 14:00:52.352144 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.352122    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b"
Jul 15 14:00:52.354914 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.354893    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 14:00:52.362034 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.362008    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b"]
Jul 15 14:00:52.455475 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.455436    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b\" (UID: \"23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b"
Jul 15 14:00:52.556258 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.556216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b\" (UID: \"23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b"
Jul 15 14:00:52.556853 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.556832    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b\" (UID: \"23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b"
Jul 15 14:00:52.664012 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.663925    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b"
Jul 15 14:00:52.805466 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:52.805438    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b"]
Jul 15 14:00:52.808235 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:00:52.808207    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a6eea2_f7c6_4fe1_b4b5_f9456a4508f4.slice/crio-d834dfcf43d50b726f6b56eff33fe12b981a7f1b282e688e3418efa02205ebb8 WatchSource:0}: Error finding container d834dfcf43d50b726f6b56eff33fe12b981a7f1b282e688e3418efa02205ebb8: Status 404 returned error can't find the container with id d834dfcf43d50b726f6b56eff33fe12b981a7f1b282e688e3418efa02205ebb8
Jul 15 14:00:53.623308 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:53.623253    2577 generic.go:356] "Generic (PLEG): container finished" podID="23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4" containerID="3a072e989208259ddfe072e1f451a71d24c9ecc59337a4336342c2ce54c7c924" exitCode=0
Jul 15 14:00:53.623308 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:53.623296    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b" event={"ID":"23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4","Type":"ContainerDied","Data":"3a072e989208259ddfe072e1f451a71d24c9ecc59337a4336342c2ce54c7c924"}
Jul 15 14:00:53.623783 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:53.623362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b" event={"ID":"23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4","Type":"ContainerStarted","Data":"d834dfcf43d50b726f6b56eff33fe12b981a7f1b282e688e3418efa02205ebb8"}
Jul 15 14:00:54.773248 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:54.773218    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b"
Jul 15 14:00:54.874307 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:54.874257    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4-maas-api-service-ca\") pod \"23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4\" (UID: \"23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4\") "
Jul 15 14:00:54.874630 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:54.874606    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4" (UID: "23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:00:54.975249 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:54.975166    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:00:55.634629 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:55.634602    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b"
Jul 15 14:00:55.634803 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:55.634601    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b" event={"ID":"23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4","Type":"ContainerDied","Data":"d834dfcf43d50b726f6b56eff33fe12b981a7f1b282e688e3418efa02205ebb8"}
Jul 15 14:00:55.634803 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:00:55.634726    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d834dfcf43d50b726f6b56eff33fe12b981a7f1b282e688e3418efa02205ebb8"
Jul 15 14:01:02.840453 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:02.840407    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"]
Jul 15 14:01:02.840946 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:02.840698    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq" podUID="67684a2d-b3d7-4e74-a4ea-a8a7bb421196" containerName="maas-api" containerID="cri-o://ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75" gracePeriod=30
Jul 15 14:01:02.866463 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:02.866427    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p"]
Jul 15 14:01:02.869739 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:02.869707    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-6ff521e6-29735400-srh2p"]
Jul 15 14:01:02.870294 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:02.870274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:03.094738 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.094680    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 14:01:03.144969 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.144933    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnwjf\" (UniqueName: \"kubernetes.io/projected/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-kube-api-access-qnwjf\") pod \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") "
Jul 15 14:01:03.145121 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.145006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-openshift-service-ca\") pod \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") "
Jul 15 14:01:03.145121 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.145053    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-maas-api-tls\") pod \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\" (UID: \"67684a2d-b3d7-4e74-a4ea-a8a7bb421196\") "
Jul 15 14:01:03.145390 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.145361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67684a2d-b3d7-4e74-a4ea-a8a7bb421196" (UID: "67684a2d-b3d7-4e74-a4ea-a8a7bb421196"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:01:03.147258 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.147228    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-kube-api-access-qnwjf" (OuterVolumeSpecName: "kube-api-access-qnwjf") pod "67684a2d-b3d7-4e74-a4ea-a8a7bb421196" (UID: "67684a2d-b3d7-4e74-a4ea-a8a7bb421196"). InnerVolumeSpecName "kube-api-access-qnwjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:01:03.147359 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.147331    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67684a2d-b3d7-4e74-a4ea-a8a7bb421196" (UID: "67684a2d-b3d7-4e74-a4ea-a8a7bb421196"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:01:03.246454 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.246414    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:01:03.246454 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.246447    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnwjf\" (UniqueName: \"kubernetes.io/projected/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-kube-api-access-qnwjf\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:01:03.246454 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.246462    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67684a2d-b3d7-4e74-a4ea-a8a7bb421196-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:01:03.673541 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.673500    2577 generic.go:356] "Generic (PLEG): container finished" podID="67684a2d-b3d7-4e74-a4ea-a8a7bb421196" containerID="ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75" exitCode=0
Jul 15 14:01:03.673708 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.673549    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"
Jul 15 14:01:03.673708 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.673550    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq" event={"ID":"67684a2d-b3d7-4e74-a4ea-a8a7bb421196","Type":"ContainerDied","Data":"ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75"}
Jul 15 14:01:03.673708 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.673605    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq" event={"ID":"67684a2d-b3d7-4e74-a4ea-a8a7bb421196","Type":"ContainerDied","Data":"a9b4bd2dc98d56f8ac3a087ec6af97b9ed17c02fa48a70ee770fd11631b68cdd"}
Jul 15 14:01:03.673708 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.673640    2577 scope.go:117] "RemoveContainer" containerID="ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75"
Jul 15 14:01:03.683116 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.683092    2577 scope.go:117] "RemoveContainer" containerID="ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75"
Jul 15 14:01:03.683435 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:01:03.683415    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75\": container with ID starting with ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75 not found: ID does not exist" containerID="ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75"
Jul 15 14:01:03.683501 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.683459    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75"} err="failed to get container status \"ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75\": rpc error: code = NotFound desc = could not find container \"ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75\": container with ID starting with ceefad0a495ccb5c2de081e19049ad6355ec094760b0925257a25eadfd00cc75 not found: ID does not exist"
Jul 15 14:01:03.696542 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.696514    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"]
Jul 15 14:01:03.703279 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:03.703226    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6ff521e6-68846875cb-9c2hq"]
Jul 15 14:01:04.147034 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:04.146998    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67684a2d-b3d7-4e74-a4ea-a8a7bb421196" path="/var/lib/kubelet/pods/67684a2d-b3d7-4e74-a4ea-a8a7bb421196/volumes"
Jul 15 14:01:04.147429 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:04.147322    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92e6d1e4-51c2-478e-af8c-f0095c746405" path="/var/lib/kubelet/pods/92e6d1e4-51c2-478e-af8c-f0095c746405/volumes"
Jul 15 14:01:07.523520 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:07.523487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:14.164123 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:14.164081    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:14.238808 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:14.238775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:17.755815 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:17.755778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:18.480848 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:18.480782    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:18.589219 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:18.589113    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:18.657162 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:18.657130    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:20.328355 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:20.328298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:26.457670 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:26.457632    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:26.576598 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:26.576563    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:26.661372 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:26.661339    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:26.761719 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:26.761694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:27.056600 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:27.056515    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:27.132077 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:27.132037    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:27.241973 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:27.241937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:29.164696 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:29.164658    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:53.262400 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:53.262367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:01:59.976532 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:01:59.976501    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:00.088975 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:00.088941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:00.313361 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:00.313321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:16.138886 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:16.138815    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:16.288551 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:16.288520    2577 scope.go:117] "RemoveContainer" containerID="a47a2cf3950dccab011554517e680219d78fc3da981180df0a00a83fb8895f36"
Jul 15 14:02:22.873495 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:22.873452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:22.989497 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:22.989457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:23.084664 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:23.084631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:33.684753 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:33.684715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:34.556992 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:34.556958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:40.715997 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:40.715959    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:40.827640 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:40.827599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:40.929200 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:40.929163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:41.030798 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:41.030761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:41.880680 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:41.880647    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:41.967207 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:41.967175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:42.075130 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:42.075095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:43.841818 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:43.841786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:50.643844 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:50.643813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:50.754103 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:50.754074    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:51.300848 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:51.300811    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:57.615021 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:57.614984    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:57.723647 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:57.723612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:57.850838 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:57.850801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:58.240821 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:58.240792    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:02:58.399664 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:02:58.399630    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:01.866209 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.866169    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"]
Jul 15 14:03:01.866651 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.866535    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67684a2d-b3d7-4e74-a4ea-a8a7bb421196" containerName="maas-api"
Jul 15 14:03:01.866651 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.866547    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="67684a2d-b3d7-4e74-a4ea-a8a7bb421196" containerName="maas-api"
Jul 15 14:03:01.866651 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.866556    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerName="revoke-keys"
Jul 15 14:03:01.866651 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.866561    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerName="revoke-keys"
Jul 15 14:03:01.866651 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.866568    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4" containerName="revoke-keys"
Jul 15 14:03:01.866651 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.866573    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4" containerName="revoke-keys"
Jul 15 14:03:01.866651 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.866626    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e06e2c5-7957-48ed-88f5-36b73e5f452e" containerName="revoke-keys"
Jul 15 14:03:01.866651 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.866635    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4" containerName="revoke-keys"
Jul 15 14:03:01.866651 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.866642    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="67684a2d-b3d7-4e74-a4ea-a8a7bb421196" containerName="maas-api"
Jul 15 14:03:01.869747 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.869729    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:01.874474 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.874451    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c382ffa7-serving-cert\""
Jul 15 14:03:01.885698 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.885646    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"]
Jul 15 14:03:01.968209 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.968175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac397dd5-2eca-4941-903f-feea55fe0c47-maas-api-tls\") pod \"maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:01.968396 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.968220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kp2p\" (UniqueName: \"kubernetes.io/projected/ac397dd5-2eca-4941-903f-feea55fe0c47-kube-api-access-9kp2p\") pod \"maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:01.968396 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:01.968247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac397dd5-2eca-4941-903f-feea55fe0c47-openshift-service-ca\") pod \"maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:02.069334 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.069255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac397dd5-2eca-4941-903f-feea55fe0c47-maas-api-tls\") pod \"maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:02.069550 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.069346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kp2p\" (UniqueName: \"kubernetes.io/projected/ac397dd5-2eca-4941-903f-feea55fe0c47-kube-api-access-9kp2p\") pod \"maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:02.069550 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.069373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac397dd5-2eca-4941-903f-feea55fe0c47-openshift-service-ca\") pod \"maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:02.069550 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:03:02.069426    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-serving-cert: secret "maas-api-e2e-mt-c382ffa7-serving-cert" not found
Jul 15 14:03:02.069550 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:03:02.069518    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac397dd5-2eca-4941-903f-feea55fe0c47-maas-api-tls podName:ac397dd5-2eca-4941-903f-feea55fe0c47 nodeName:}" failed. No retries permitted until 2026-07-15 14:03:02.569503787 +0000 UTC m=+2026.912438845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac397dd5-2eca-4941-903f-feea55fe0c47-maas-api-tls") pod "maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7" (UID: "ac397dd5-2eca-4941-903f-feea55fe0c47") : secret "maas-api-e2e-mt-c382ffa7-serving-cert" not found
Jul 15 14:03:02.069980 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.069964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac397dd5-2eca-4941-903f-feea55fe0c47-openshift-service-ca\") pod \"maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:02.090527 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.090496    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kp2p\" (UniqueName: \"kubernetes.io/projected/ac397dd5-2eca-4941-903f-feea55fe0c47-kube-api-access-9kp2p\") pod \"maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:02.093110 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.093087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:02.184658 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.184578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:02.288392 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.288339    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:02.574445 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.574407    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac397dd5-2eca-4941-903f-feea55fe0c47-maas-api-tls\") pod \"maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:02.577236 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.577210    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac397dd5-2eca-4941-903f-feea55fe0c47-maas-api-tls\") pod \"maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:02.780020 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.779979    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:02.911727 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.911681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:02.921660 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.921504    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"]
Jul 15 14:03:02.924085 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:03:02.924052    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac397dd5_2eca_4941_903f_feea55fe0c47.slice/crio-89ce8dacdbf27dc65fd3314aa8c9640cc9ab890518aa9eecde343abe19afc652 WatchSource:0}: Error finding container 89ce8dacdbf27dc65fd3314aa8c9640cc9ab890518aa9eecde343abe19afc652: Status 404 returned error can't find the container with id 89ce8dacdbf27dc65fd3314aa8c9640cc9ab890518aa9eecde343abe19afc652
Jul 15 14:03:02.925427 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:02.925410    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 14:03:03.059143 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:03.059115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:03.256959 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:03.256929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7" event={"ID":"ac397dd5-2eca-4941-903f-feea55fe0c47","Type":"ContainerStarted","Data":"89ce8dacdbf27dc65fd3314aa8c9640cc9ab890518aa9eecde343abe19afc652"}
Jul 15 14:03:04.262794 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:04.262763    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7" event={"ID":"ac397dd5-2eca-4941-903f-feea55fe0c47","Type":"ContainerStarted","Data":"4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98"}
Jul 15 14:03:04.263198 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:04.262891    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:04.283100 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:04.283043    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7" podStartSLOduration=2.628018231 podStartE2EDuration="3.283027077s" totalImagesPullingTime="655.008846ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:03:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:03:04.280979002 +0000 UTC m=+2028.623914107" watchObservedRunningTime="2026-07-15 14:03:04.283027077 +0000 UTC m=+2028.625962201"
Jul 15 14:03:06.155966 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:06.155907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:06.935954 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:06.935915    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:07.028451 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:07.028419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:07.441619 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:07.441586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:10.274631 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:10.274602    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:13.629045 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:13.629000    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:13.728490 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:13.728458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:14.041753 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:14.041714    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98"]
Jul 15 14:03:14.045363 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:14.045344    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98"
Jul 15 14:03:14.049738 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:14.049711    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 14:03:14.062523 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:14.062493    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98"]
Jul 15 14:03:14.178826 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:14.178790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e5d016-26de-496c-9c73-bea3d7a1dc70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98\" (UID: \"d2e5d016-26de-496c-9c73-bea3d7a1dc70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98"
Jul 15 14:03:14.279764 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:14.279720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e5d016-26de-496c-9c73-bea3d7a1dc70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98\" (UID: \"d2e5d016-26de-496c-9c73-bea3d7a1dc70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98"
Jul 15 14:03:14.280364 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:14.280344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e5d016-26de-496c-9c73-bea3d7a1dc70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98\" (UID: \"d2e5d016-26de-496c-9c73-bea3d7a1dc70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98"
Jul 15 14:03:14.355727 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:14.355640    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98"
Jul 15 14:03:14.493982 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:14.493957    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98"]
Jul 15 14:03:14.496136 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:03:14.496109    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e5d016_26de_496c_9c73_bea3d7a1dc70.slice/crio-7cb245cb7843e8fdd6af3887ca70e166f3a2a675e2b05c96c72461fcca3a101b WatchSource:0}: Error finding container 7cb245cb7843e8fdd6af3887ca70e166f3a2a675e2b05c96c72461fcca3a101b: Status 404 returned error can't find the container with id 7cb245cb7843e8fdd6af3887ca70e166f3a2a675e2b05c96c72461fcca3a101b
Jul 15 14:03:15.322809 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:15.322775    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2e5d016-26de-496c-9c73-bea3d7a1dc70" containerID="54571bddc8706282359713d9863241c5f346ac768044024355de8a9d878adf72" exitCode=0
Jul 15 14:03:15.323246 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:15.322869    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98" event={"ID":"d2e5d016-26de-496c-9c73-bea3d7a1dc70","Type":"ContainerDied","Data":"54571bddc8706282359713d9863241c5f346ac768044024355de8a9d878adf72"}
Jul 15 14:03:15.323246 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:15.322937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98" event={"ID":"d2e5d016-26de-496c-9c73-bea3d7a1dc70","Type":"ContainerStarted","Data":"7cb245cb7843e8fdd6af3887ca70e166f3a2a675e2b05c96c72461fcca3a101b"}
Jul 15 14:03:16.462330 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:16.462307    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98"
Jul 15 14:03:16.605404 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:16.605315    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e5d016-26de-496c-9c73-bea3d7a1dc70-maas-api-service-ca\") pod \"d2e5d016-26de-496c-9c73-bea3d7a1dc70\" (UID: \"d2e5d016-26de-496c-9c73-bea3d7a1dc70\") "
Jul 15 14:03:16.605678 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:16.605653    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e5d016-26de-496c-9c73-bea3d7a1dc70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d2e5d016-26de-496c-9c73-bea3d7a1dc70" (UID: "d2e5d016-26de-496c-9c73-bea3d7a1dc70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:03:16.706191 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:16.706152    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e5d016-26de-496c-9c73-bea3d7a1dc70-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:03:17.333237 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:17.333204    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98"
Jul 15 14:03:17.333237 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:17.333211    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98" event={"ID":"d2e5d016-26de-496c-9c73-bea3d7a1dc70","Type":"ContainerDied","Data":"7cb245cb7843e8fdd6af3887ca70e166f3a2a675e2b05c96c72461fcca3a101b"}
Jul 15 14:03:17.333465 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:17.333256    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cb245cb7843e8fdd6af3887ca70e166f3a2a675e2b05c96c72461fcca3a101b"
Jul 15 14:03:24.623487 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.623454    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"]
Jul 15 14:03:24.624031 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.623708    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7" podUID="ac397dd5-2eca-4941-903f-feea55fe0c47" containerName="maas-api" containerID="cri-o://4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98" gracePeriod=30
Jul 15 14:03:24.660871 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.660838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:24.869645 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.869616    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:24.877786 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.877731    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac397dd5-2eca-4941-903f-feea55fe0c47-openshift-service-ca\") pod \"ac397dd5-2eca-4941-903f-feea55fe0c47\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") "
Jul 15 14:03:24.877786 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.877762    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kp2p\" (UniqueName: \"kubernetes.io/projected/ac397dd5-2eca-4941-903f-feea55fe0c47-kube-api-access-9kp2p\") pod \"ac397dd5-2eca-4941-903f-feea55fe0c47\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") "
Jul 15 14:03:24.877786 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.877786    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac397dd5-2eca-4941-903f-feea55fe0c47-maas-api-tls\") pod \"ac397dd5-2eca-4941-903f-feea55fe0c47\" (UID: \"ac397dd5-2eca-4941-903f-feea55fe0c47\") "
Jul 15 14:03:24.878079 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.878054    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac397dd5-2eca-4941-903f-feea55fe0c47-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ac397dd5-2eca-4941-903f-feea55fe0c47" (UID: "ac397dd5-2eca-4941-903f-feea55fe0c47"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:03:24.879991 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.879966    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac397dd5-2eca-4941-903f-feea55fe0c47-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ac397dd5-2eca-4941-903f-feea55fe0c47" (UID: "ac397dd5-2eca-4941-903f-feea55fe0c47"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:03:24.879991 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.879972    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac397dd5-2eca-4941-903f-feea55fe0c47-kube-api-access-9kp2p" (OuterVolumeSpecName: "kube-api-access-9kp2p") pod "ac397dd5-2eca-4941-903f-feea55fe0c47" (UID: "ac397dd5-2eca-4941-903f-feea55fe0c47"). InnerVolumeSpecName "kube-api-access-9kp2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:03:24.979384 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.979348    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac397dd5-2eca-4941-903f-feea55fe0c47-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:03:24.979384 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.979377    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac397dd5-2eca-4941-903f-feea55fe0c47-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:03:24.979384 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:24.979387    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kp2p\" (UniqueName: \"kubernetes.io/projected/ac397dd5-2eca-4941-903f-feea55fe0c47-kube-api-access-9kp2p\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:03:25.374157 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:25.374118    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac397dd5-2eca-4941-903f-feea55fe0c47" containerID="4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98" exitCode=0
Jul 15 14:03:25.374157 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:25.374159    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"
Jul 15 14:03:25.374405 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:25.374203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7" event={"ID":"ac397dd5-2eca-4941-903f-feea55fe0c47","Type":"ContainerDied","Data":"4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98"}
Jul 15 14:03:25.374405 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:25.374256    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7" event={"ID":"ac397dd5-2eca-4941-903f-feea55fe0c47","Type":"ContainerDied","Data":"89ce8dacdbf27dc65fd3314aa8c9640cc9ab890518aa9eecde343abe19afc652"}
Jul 15 14:03:25.374405 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:25.374303    2577 scope.go:117] "RemoveContainer" containerID="4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98"
Jul 15 14:03:25.383455 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:25.383438    2577 scope.go:117] "RemoveContainer" containerID="4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98"
Jul 15 14:03:25.383709 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:03:25.383686    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98\": container with ID starting with 4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98 not found: ID does not exist" containerID="4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98"
Jul 15 14:03:25.383774 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:25.383717    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98"} err="failed to get container status \"4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98\": rpc error: code = NotFound desc = could not find container \"4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98\": container with ID starting with 4e9fd2bd13fb41d956daf4270e844e2585eb17e6deb9cad9588ad2a33bc76c98 not found: ID does not exist"
Jul 15 14:03:25.400180 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:25.400136    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"]
Jul 15 14:03:25.402104 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:25.402071    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c382ffa7-5b64764749-r2sz7"]
Jul 15 14:03:26.152108 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:26.152071    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac397dd5-2eca-4941-903f-feea55fe0c47" path="/var/lib/kubelet/pods/ac397dd5-2eca-4941-903f-feea55fe0c47/volumes"
Jul 15 14:03:29.151492 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:29.151460    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:35.441470 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:35.441428    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:35.542881 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:35.542847    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:35.695563 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:35.695470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:35.763376 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:35.763342    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:39.856027 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:39.855941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:40.105584 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:40.105551    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:40.228514 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:40.228421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:40.327083 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:40.327055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:40.508821 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:40.508793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:46.677294 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:46.677232    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:46.788293 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:46.788233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:46.938553 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:46.938475    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:47.110023 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:47.109988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:53.308701 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:53.308672    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:03:53.413549 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:03:53.413520    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:00.203363 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:00.203329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:00.574773 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:00.574736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:00.925009 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:00.924923    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:03.887147 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:03.887061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:03.975561 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:03.975529    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:04.068506 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.068469    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"]
Jul 15 14:04:04.068972 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.068955    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e5d016-26de-496c-9c73-bea3d7a1dc70" containerName="revoke-keys"
Jul 15 14:04:04.069074 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.068975    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e5d016-26de-496c-9c73-bea3d7a1dc70" containerName="revoke-keys"
Jul 15 14:04:04.069074 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.069017    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac397dd5-2eca-4941-903f-feea55fe0c47" containerName="maas-api"
Jul 15 14:04:04.069074 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.069025    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac397dd5-2eca-4941-903f-feea55fe0c47" containerName="maas-api"
Jul 15 14:04:04.069215 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.069095    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e5d016-26de-496c-9c73-bea3d7a1dc70" containerName="revoke-keys"
Jul 15 14:04:04.069215 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.069108    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac397dd5-2eca-4941-903f-feea55fe0c47" containerName="maas-api"
Jul 15 14:04:04.072584 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.072563    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.075476 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.075456    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b404f66f-serving-cert\""
Jul 15 14:04:04.082025 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.082003    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"]
Jul 15 14:04:04.085976 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.085948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:04.205351 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.205242    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-openshift-service-ca\") pod \"maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.205351 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.205299    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlv45\" (UniqueName: \"kubernetes.io/projected/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-kube-api-access-hlv45\") pod \"maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.205536 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.205369    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-maas-api-tls\") pod \"maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.296362 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.296329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:04.305809 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.305779    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-openshift-service-ca\") pod \"maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.305978 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.305814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlv45\" (UniqueName: \"kubernetes.io/projected/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-kube-api-access-hlv45\") pod \"maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.306055 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.305980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-maas-api-tls\") pod \"maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.306800 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.306771    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-openshift-service-ca\") pod \"maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.308576 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.308554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-maas-api-tls\") pod \"maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.315680 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.315656    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlv45\" (UniqueName: \"kubernetes.io/projected/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-kube-api-access-hlv45\") pod \"maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.385144 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.385100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:04.525970 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.525908    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"]
Jul 15 14:04:04.528553 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:04:04.528531    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd56ac8_bebd_4f76_81d1_5c553d34ef68.slice/crio-8899825d2edab7f5210189aee446417db620103588869298cc90a019e5a172c9 WatchSource:0}: Error finding container 8899825d2edab7f5210189aee446417db620103588869298cc90a019e5a172c9: Status 404 returned error can't find the container with id 8899825d2edab7f5210189aee446417db620103588869298cc90a019e5a172c9
Jul 15 14:04:04.563334 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.563300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp" event={"ID":"fdd56ac8-bebd-4f76-81d1-5c553d34ef68","Type":"ContainerStarted","Data":"8899825d2edab7f5210189aee446417db620103588869298cc90a019e5a172c9"}
Jul 15 14:04:04.760649 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.760620    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:04.820163 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:04.820130    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:05.569798 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:05.569760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp" event={"ID":"fdd56ac8-bebd-4f76-81d1-5c553d34ef68","Type":"ContainerStarted","Data":"d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386"}
Jul 15 14:04:05.570222 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:05.569839    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:05.593025 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:05.592966    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp" podStartSLOduration=1.119739757 podStartE2EDuration="1.592949761s" totalImagesPullingTime="473.210004ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:04:05.590841645 +0000 UTC m=+2089.933776739" watchObservedRunningTime="2026-07-15 14:04:05.592949761 +0000 UTC m=+2089.935884840"
Jul 15 14:04:08.428559 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:08.428456    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:11.580852 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:11.580821    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:11.883984 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:11.883889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:16.205497 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:16.205466    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 14:04:16.210276 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:16.210232    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 14:04:16.315718 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:16.315687    2577 scope.go:117] "RemoveContainer" containerID="1d280b81f5180ec2f2436fa6560b5923833d381efa944f1e8c4b366932a92cfc"
Jul 15 14:04:33.068850 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:33.068810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:36.672951 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:36.672917    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj"]
Jul 15 14:04:36.676481 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:36.676463    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj"
Jul 15 14:04:36.679225 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:36.679200    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 14:04:36.685029 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:36.685006    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj"]
Jul 15 14:04:36.770845 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:36.770805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/156ef1ea-8905-4016-b13d-acba86b69628-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj\" (UID: \"156ef1ea-8905-4016-b13d-acba86b69628\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj"
Jul 15 14:04:36.871644 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:36.871589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/156ef1ea-8905-4016-b13d-acba86b69628-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj\" (UID: \"156ef1ea-8905-4016-b13d-acba86b69628\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj"
Jul 15 14:04:36.872211 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:36.872189    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/156ef1ea-8905-4016-b13d-acba86b69628-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj\" (UID: \"156ef1ea-8905-4016-b13d-acba86b69628\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj"
Jul 15 14:04:36.987550 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:36.987450    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj"
Jul 15 14:04:37.128388 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:37.128348    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj"]
Jul 15 14:04:37.131233 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:04:37.131207    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod156ef1ea_8905_4016_b13d_acba86b69628.slice/crio-e906ba2c36cd633477c2eaffe0db93b4e54225ef8e58a03c0e57c6ab28b2de0f WatchSource:0}: Error finding container e906ba2c36cd633477c2eaffe0db93b4e54225ef8e58a03c0e57c6ab28b2de0f: Status 404 returned error can't find the container with id e906ba2c36cd633477c2eaffe0db93b4e54225ef8e58a03c0e57c6ab28b2de0f
Jul 15 14:04:37.726412 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:37.726379    2577 generic.go:356] "Generic (PLEG): container finished" podID="156ef1ea-8905-4016-b13d-acba86b69628" containerID="6cbc6185a5b1d4f39db5b92c8d1d6c34b2d512f920437c23bd7c304ebcc018da" exitCode=0
Jul 15 14:04:37.726765 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:37.726458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj" event={"ID":"156ef1ea-8905-4016-b13d-acba86b69628","Type":"ContainerDied","Data":"6cbc6185a5b1d4f39db5b92c8d1d6c34b2d512f920437c23bd7c304ebcc018da"}
Jul 15 14:04:37.726765 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:37.726518    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj" event={"ID":"156ef1ea-8905-4016-b13d-acba86b69628","Type":"ContainerStarted","Data":"e906ba2c36cd633477c2eaffe0db93b4e54225ef8e58a03c0e57c6ab28b2de0f"}
Jul 15 14:04:38.872145 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:38.872119    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj"
Jul 15 14:04:38.886208 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:38.886176    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/156ef1ea-8905-4016-b13d-acba86b69628-maas-api-service-ca\") pod \"156ef1ea-8905-4016-b13d-acba86b69628\" (UID: \"156ef1ea-8905-4016-b13d-acba86b69628\") "
Jul 15 14:04:38.886588 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:38.886567    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/156ef1ea-8905-4016-b13d-acba86b69628-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "156ef1ea-8905-4016-b13d-acba86b69628" (UID: "156ef1ea-8905-4016-b13d-acba86b69628"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:04:38.986900 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:38.986805    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/156ef1ea-8905-4016-b13d-acba86b69628-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:04:39.738019 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:39.737988    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj"
Jul 15 14:04:39.738200 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:39.737990    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj" event={"ID":"156ef1ea-8905-4016-b13d-acba86b69628","Type":"ContainerDied","Data":"e906ba2c36cd633477c2eaffe0db93b4e54225ef8e58a03c0e57c6ab28b2de0f"}
Jul 15 14:04:39.738200 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:39.738116    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e906ba2c36cd633477c2eaffe0db93b4e54225ef8e58a03c0e57c6ab28b2de0f"
Jul 15 14:04:47.244911 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.244870    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"]
Jul 15 14:04:47.245319 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.245079    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp" podUID="fdd56ac8-bebd-4f76-81d1-5c553d34ef68" containerName="maas-api" containerID="cri-o://d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386" gracePeriod=30
Jul 15 14:04:47.279865 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.279837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:47.513774 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.513748    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:47.549601 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.549564    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlv45\" (UniqueName: \"kubernetes.io/projected/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-kube-api-access-hlv45\") pod \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") "
Jul 15 14:04:47.549798 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.549608    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-maas-api-tls\") pod \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") "
Jul 15 14:04:47.549798 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.549657    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-openshift-service-ca\") pod \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\" (UID: \"fdd56ac8-bebd-4f76-81d1-5c553d34ef68\") "
Jul 15 14:04:47.550150 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.550112    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fdd56ac8-bebd-4f76-81d1-5c553d34ef68" (UID: "fdd56ac8-bebd-4f76-81d1-5c553d34ef68"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:04:47.551913 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.551882    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fdd56ac8-bebd-4f76-81d1-5c553d34ef68" (UID: "fdd56ac8-bebd-4f76-81d1-5c553d34ef68"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:04:47.552013 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.551936    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-kube-api-access-hlv45" (OuterVolumeSpecName: "kube-api-access-hlv45") pod "fdd56ac8-bebd-4f76-81d1-5c553d34ef68" (UID: "fdd56ac8-bebd-4f76-81d1-5c553d34ef68"). InnerVolumeSpecName "kube-api-access-hlv45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:04:47.650235 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.650191    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:04:47.650235 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.650229    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlv45\" (UniqueName: \"kubernetes.io/projected/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-kube-api-access-hlv45\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:04:47.650235 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.650244    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdd56ac8-bebd-4f76-81d1-5c553d34ef68-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:04:47.777333 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.777303    2577 generic.go:356] "Generic (PLEG): container finished" podID="fdd56ac8-bebd-4f76-81d1-5c553d34ef68" containerID="d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386" exitCode=0
Jul 15 14:04:47.777523 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.777342    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"
Jul 15 14:04:47.777523 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.777373    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp" event={"ID":"fdd56ac8-bebd-4f76-81d1-5c553d34ef68","Type":"ContainerDied","Data":"d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386"}
Jul 15 14:04:47.777523 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.777415    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp" event={"ID":"fdd56ac8-bebd-4f76-81d1-5c553d34ef68","Type":"ContainerDied","Data":"8899825d2edab7f5210189aee446417db620103588869298cc90a019e5a172c9"}
Jul 15 14:04:47.777523 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.777437    2577 scope.go:117] "RemoveContainer" containerID="d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386"
Jul 15 14:04:47.786970 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.786948    2577 scope.go:117] "RemoveContainer" containerID="d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386"
Jul 15 14:04:47.787308 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:04:47.787286    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386\": container with ID starting with d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386 not found: ID does not exist" containerID="d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386"
Jul 15 14:04:47.787361 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.787320    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386"} err="failed to get container status \"d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386\": rpc error: code = NotFound desc = could not find container \"d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386\": container with ID starting with d42ff4967d71233544a65ce7c5f28e1bada0212fa62ca92bfa76a4278a6a0386 not found: ID does not exist"
Jul 15 14:04:47.799342 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.799314    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"]
Jul 15 14:04:47.802566 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:47.802544    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b404f66f-84f46b6b7f-zt6xp"]
Jul 15 14:04:48.147431 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:48.147352    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdd56ac8-bebd-4f76-81d1-5c553d34ef68" path="/var/lib/kubelet/pods/fdd56ac8-bebd-4f76-81d1-5c553d34ef68/volumes"
Jul 15 14:04:51.768196 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:51.768168    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:56.835949 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:56.835913    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:59.501874 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:59.501828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:04:59.571852 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:04:59.571819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:16.338672 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:16.338638    2577 scope.go:117] "RemoveContainer" containerID="1e5e8f54a14f40069cba43667085ecd4a041caf707f86f727f1a5f4b002a2870"
Jul 15 14:05:19.028505 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:19.028476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:20.564872 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:20.564836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:25.035992 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:25.035954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:25.126965 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:25.126927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:34.073502 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:34.073440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:34.317170 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:34.317139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:35.432860 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:35.432831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:35.529076 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:35.529036    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:35.619032 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:35.618998    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:36.376188 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:36.376144    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:36.469910 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:36.469878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:36.574580 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:36.574545    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:38.020219 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:38.020191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:44.255481 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:44.255435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:44.354899 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:44.354864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:45.019665 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:45.019633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:50.952252 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:50.952220    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:51.052403 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:51.052371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:51.158640 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:51.158610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:05:51.255384 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:05:51.255335    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:00.750872 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:00.750820    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:01.744681 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:01.744643    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:02.124407 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:02.124374    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:16.360770 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:16.360737    2577 scope.go:117] "RemoveContainer" containerID="6c62fa95a79d8484581bc093cd70ddea6f712fddd8b1391db0f983fe90a1eca1"
Jul 15 14:06:16.370635 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:16.370612    2577 scope.go:117] "RemoveContainer" containerID="b609a720e17ad87c425604630420a0eae94d44e2d080d38c1166f6fea76581e6"
Jul 15 14:06:23.173473 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:23.173427    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:29.149909 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:29.149831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:29.248275 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:29.248242    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:29.468387 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:29.468293    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:29.792126 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:29.792091    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:30.083332 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:30.083193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:32.673772 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.673729    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:32.771536 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.771500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:32.858886 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.858851    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"]
Jul 15 14:06:32.859407 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.859390    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdd56ac8-bebd-4f76-81d1-5c553d34ef68" containerName="maas-api"
Jul 15 14:06:32.859472 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.859411    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdd56ac8-bebd-4f76-81d1-5c553d34ef68" containerName="maas-api"
Jul 15 14:06:32.859472 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.859435    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="156ef1ea-8905-4016-b13d-acba86b69628" containerName="revoke-keys"
Jul 15 14:06:32.859472 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.859443    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="156ef1ea-8905-4016-b13d-acba86b69628" containerName="revoke-keys"
Jul 15 14:06:32.859569 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.859537    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="156ef1ea-8905-4016-b13d-acba86b69628" containerName="revoke-keys"
Jul 15 14:06:32.859569 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.859551    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdd56ac8-bebd-4f76-81d1-5c553d34ef68" containerName="maas-api"
Jul 15 14:06:32.863019 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.862997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:32.865993 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.865971    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-16550d-serving-cert\""
Jul 15 14:06:32.878508 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.878468    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"]
Jul 15 14:06:32.889282 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.889232    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:32.927642 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.927540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82c3ed4d-14c8-4574-9577-68f1006d5cb1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:32.927642 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.927589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc4j8\" (UniqueName: \"kubernetes.io/projected/82c3ed4d-14c8-4574-9577-68f1006d5cb1-kube-api-access-gc4j8\") pod \"maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:32.927845 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:32.927655    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82c3ed4d-14c8-4574-9577-68f1006d5cb1-maas-api-tls\") pod \"maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:33.028770 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.028728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82c3ed4d-14c8-4574-9577-68f1006d5cb1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:33.028770 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.028772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc4j8\" (UniqueName: \"kubernetes.io/projected/82c3ed4d-14c8-4574-9577-68f1006d5cb1-kube-api-access-gc4j8\") pod \"maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:33.029036 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.028816    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82c3ed4d-14c8-4574-9577-68f1006d5cb1-maas-api-tls\") pod \"maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:33.029036 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:06:33.028935    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-serving-cert: secret "maas-api-e2e-inf-a-16550d-serving-cert" not found
Jul 15 14:06:33.029036 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:06:33.029018    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c3ed4d-14c8-4574-9577-68f1006d5cb1-maas-api-tls podName:82c3ed4d-14c8-4574-9577-68f1006d5cb1 nodeName:}" failed. No retries permitted until 2026-07-15 14:06:33.52899698 +0000 UTC m=+2237.871932052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/82c3ed4d-14c8-4574-9577-68f1006d5cb1-maas-api-tls") pod "maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv" (UID: "82c3ed4d-14c8-4574-9577-68f1006d5cb1") : secret "maas-api-e2e-inf-a-16550d-serving-cert" not found
Jul 15 14:06:33.029572 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.029549    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82c3ed4d-14c8-4574-9577-68f1006d5cb1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:33.038513 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.038489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc4j8\" (UniqueName: \"kubernetes.io/projected/82c3ed4d-14c8-4574-9577-68f1006d5cb1-kube-api-access-gc4j8\") pod \"maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:33.391906 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.391858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:33.533661 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.533625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82c3ed4d-14c8-4574-9577-68f1006d5cb1-maas-api-tls\") pod \"maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:33.536497 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.536470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82c3ed4d-14c8-4574-9577-68f1006d5cb1-maas-api-tls\") pod \"maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:33.587333 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.587296    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:33.691906 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.691816    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:33.775058 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.775010    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:33.928091 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:33.928059    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"]
Jul 15 14:06:33.929567 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:06:33.929535    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c3ed4d_14c8_4574_9577_68f1006d5cb1.slice/crio-a0d84879a8212f7163382cc43dbe87978ac25a731f88a2899b2fe3b59eecdbbf WatchSource:0}: Error finding container a0d84879a8212f7163382cc43dbe87978ac25a731f88a2899b2fe3b59eecdbbf: Status 404 returned error can't find the container with id a0d84879a8212f7163382cc43dbe87978ac25a731f88a2899b2fe3b59eecdbbf
Jul 15 14:06:34.304935 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:34.304902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv" event={"ID":"82c3ed4d-14c8-4574-9577-68f1006d5cb1","Type":"ContainerStarted","Data":"a0d84879a8212f7163382cc43dbe87978ac25a731f88a2899b2fe3b59eecdbbf"}
Jul 15 14:06:35.312120 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:35.312083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv" event={"ID":"82c3ed4d-14c8-4574-9577-68f1006d5cb1","Type":"ContainerStarted","Data":"8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261"}
Jul 15 14:06:35.312550 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:35.312234    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:35.341250 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:35.341189    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv" podStartSLOduration=2.7675886480000003 podStartE2EDuration="3.341171544s" totalImagesPullingTime="573.582896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:06:35.33902131 +0000 UTC m=+2239.681956434" watchObservedRunningTime="2026-07-15 14:06:35.341171544 +0000 UTC m=+2239.684106624"
Jul 15 14:06:37.272794 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:37.272718    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:37.608774 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:37.608675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:37.940484 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:37.940400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:38.107209 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:38.107170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:40.880572 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:40.880533    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:40.972785 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:40.972747    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"]
Jul 15 14:06:40.976172 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:40.976155    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:40.980006 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:40.979986    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-59923e-serving-cert\""
Jul 15 14:06:40.986209 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:40.986179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:40.995869 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:40.995842    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"]
Jul 15 14:06:41.076946 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.076905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:41.096799 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.096764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09a6f8bf-ac02-466b-9a67-40191c591922-openshift-service-ca\") pod \"maas-api-e2e-inf-b-59923e-56c8df9997-vccdt\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:41.096974 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.096808    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnn5p\" (UniqueName: \"kubernetes.io/projected/09a6f8bf-ac02-466b-9a67-40191c591922-kube-api-access-mnn5p\") pod \"maas-api-e2e-inf-b-59923e-56c8df9997-vccdt\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:41.096974 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.096873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09a6f8bf-ac02-466b-9a67-40191c591922-maas-api-tls\") pod \"maas-api-e2e-inf-b-59923e-56c8df9997-vccdt\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:41.198397 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.198298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09a6f8bf-ac02-466b-9a67-40191c591922-maas-api-tls\") pod \"maas-api-e2e-inf-b-59923e-56c8df9997-vccdt\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:41.198559 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.198416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09a6f8bf-ac02-466b-9a67-40191c591922-openshift-service-ca\") pod \"maas-api-e2e-inf-b-59923e-56c8df9997-vccdt\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:41.198559 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.198451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnn5p\" (UniqueName: \"kubernetes.io/projected/09a6f8bf-ac02-466b-9a67-40191c591922-kube-api-access-mnn5p\") pod \"maas-api-e2e-inf-b-59923e-56c8df9997-vccdt\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:41.199101 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.199081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09a6f8bf-ac02-466b-9a67-40191c591922-openshift-service-ca\") pod \"maas-api-e2e-inf-b-59923e-56c8df9997-vccdt\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:41.201056 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.201035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09a6f8bf-ac02-466b-9a67-40191c591922-maas-api-tls\") pod \"maas-api-e2e-inf-b-59923e-56c8df9997-vccdt\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:41.217747 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.217718    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnn5p\" (UniqueName: \"kubernetes.io/projected/09a6f8bf-ac02-466b-9a67-40191c591922-kube-api-access-mnn5p\") pod \"maas-api-e2e-inf-b-59923e-56c8df9997-vccdt\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:41.278971 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.278937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:41.286629 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.286611    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:41.323250 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.323214    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:06:41.444665 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.444642    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"]
Jul 15 14:06:41.446603 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:06:41.446568    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a6f8bf_ac02_466b_9a67_40191c591922.slice/crio-dac1e06f5fbc40206a83ca0dd849a298724901002b6b048d6ab4b741f1cd7570 WatchSource:0}: Error finding container dac1e06f5fbc40206a83ca0dd849a298724901002b6b048d6ab4b741f1cd7570: Status 404 returned error can't find the container with id dac1e06f5fbc40206a83ca0dd849a298724901002b6b048d6ab4b741f1cd7570
Jul 15 14:06:41.765506 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.765479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:41.864464 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:41.864432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:42.349540 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:42.349497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt" event={"ID":"09a6f8bf-ac02-466b-9a67-40191c591922","Type":"ContainerStarted","Data":"0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767"}
Jul 15 14:06:42.349917 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:42.349564    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt" event={"ID":"09a6f8bf-ac02-466b-9a67-40191c591922","Type":"ContainerStarted","Data":"dac1e06f5fbc40206a83ca0dd849a298724901002b6b048d6ab4b741f1cd7570"}
Jul 15 14:06:42.349917 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:42.349629    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:42.369658 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:42.369609    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt" podStartSLOduration=2.055540137 podStartE2EDuration="2.369592033s" totalImagesPullingTime="314.051896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:06:42.368387047 +0000 UTC m=+2246.711322163" watchObservedRunningTime="2026-07-15 14:06:42.369592033 +0000 UTC m=+2246.712527113"
Jul 15 14:06:45.393317 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:45.393253    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:48.368010 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:48.367978    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:06:54.268456 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:54.268417    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:06:54.366817 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:06:54.366780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:07:01.256257 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:01.256211    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:07:03.956571 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:03.956531    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:07:04.018331 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:04.018296    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:07:22.927302 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:22.927221    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:07:44.144198 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:44.144158    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:07:45.081818 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.081788    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:07:45.224194 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.224156    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4"]
Jul 15 14:07:45.227876 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.227857    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4"
Jul 15 14:07:45.230560 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.230539    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 14:07:45.242737 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.242710    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4"]
Jul 15 14:07:45.267275 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.267220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38412d9f-0a05-4a56-a6e4-e6a1c094e018-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4\" (UID: \"38412d9f-0a05-4a56-a6e4-e6a1c094e018\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4"
Jul 15 14:07:45.368758 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.368666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38412d9f-0a05-4a56-a6e4-e6a1c094e018-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4\" (UID: \"38412d9f-0a05-4a56-a6e4-e6a1c094e018\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4"
Jul 15 14:07:45.369328 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.369310    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38412d9f-0a05-4a56-a6e4-e6a1c094e018-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4\" (UID: \"38412d9f-0a05-4a56-a6e4-e6a1c094e018\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4"
Jul 15 14:07:45.538594 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.538557    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4"
Jul 15 14:07:45.680087 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.680063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4"]
Jul 15 14:07:45.681687 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:07:45.681653    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38412d9f_0a05_4a56_a6e4_e6a1c094e018.slice/crio-804b80f2191f5c977c7d0f31221cfa8b831bdde8a06754fe5304e066f98f56e2 WatchSource:0}: Error finding container 804b80f2191f5c977c7d0f31221cfa8b831bdde8a06754fe5304e066f98f56e2: Status 404 returned error can't find the container with id 804b80f2191f5c977c7d0f31221cfa8b831bdde8a06754fe5304e066f98f56e2
Jul 15 14:07:45.789012 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.788976    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:07:45.890907 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:45.890817    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:07:46.675342 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:46.675230    2577 generic.go:356] "Generic (PLEG): container finished" podID="38412d9f-0a05-4a56-a6e4-e6a1c094e018" containerID="10d38f81ab68f3d943b0bb9ca85ab65bb896938381e3bc27d15e685cd0f0bff8" exitCode=0
Jul 15 14:07:46.675342 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:46.675322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4" event={"ID":"38412d9f-0a05-4a56-a6e4-e6a1c094e018","Type":"ContainerDied","Data":"10d38f81ab68f3d943b0bb9ca85ab65bb896938381e3bc27d15e685cd0f0bff8"}
Jul 15 14:07:46.675764 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:46.675374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4" event={"ID":"38412d9f-0a05-4a56-a6e4-e6a1c094e018","Type":"ContainerStarted","Data":"804b80f2191f5c977c7d0f31221cfa8b831bdde8a06754fe5304e066f98f56e2"}
Jul 15 14:07:47.818164 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:47.818140    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4"
Jul 15 14:07:47.890327 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:47.890292    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38412d9f-0a05-4a56-a6e4-e6a1c094e018-maas-api-service-ca\") pod \"38412d9f-0a05-4a56-a6e4-e6a1c094e018\" (UID: \"38412d9f-0a05-4a56-a6e4-e6a1c094e018\") "
Jul 15 14:07:47.890643 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:47.890618    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38412d9f-0a05-4a56-a6e4-e6a1c094e018-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38412d9f-0a05-4a56-a6e4-e6a1c094e018" (UID: "38412d9f-0a05-4a56-a6e4-e6a1c094e018"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:07:47.991298 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:47.991207    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38412d9f-0a05-4a56-a6e4-e6a1c094e018-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:07:48.686235 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:48.686207    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4"
Jul 15 14:07:48.686428 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:48.686204    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4" event={"ID":"38412d9f-0a05-4a56-a6e4-e6a1c094e018","Type":"ContainerDied","Data":"804b80f2191f5c977c7d0f31221cfa8b831bdde8a06754fe5304e066f98f56e2"}
Jul 15 14:07:48.686428 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:48.686311    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="804b80f2191f5c977c7d0f31221cfa8b831bdde8a06754fe5304e066f98f56e2"
Jul 15 14:07:55.631693 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:55.631658    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"]
Jul 15 14:07:55.632352 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:55.631865    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv" podUID="82c3ed4d-14c8-4574-9577-68f1006d5cb1" containerName="maas-api" containerID="cri-o://8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261" gracePeriod=30
Jul 15 14:07:55.661562 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:55.661528    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:07:55.890099 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:55.890039    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:07:55.961884 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:55.961847    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82c3ed4d-14c8-4574-9577-68f1006d5cb1-maas-api-tls\") pod \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") "
Jul 15 14:07:55.962065 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:55.961911    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82c3ed4d-14c8-4574-9577-68f1006d5cb1-openshift-service-ca\") pod \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") "
Jul 15 14:07:55.962065 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:55.961940    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc4j8\" (UniqueName: \"kubernetes.io/projected/82c3ed4d-14c8-4574-9577-68f1006d5cb1-kube-api-access-gc4j8\") pod \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\" (UID: \"82c3ed4d-14c8-4574-9577-68f1006d5cb1\") "
Jul 15 14:07:55.962371 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:55.962339    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c3ed4d-14c8-4574-9577-68f1006d5cb1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "82c3ed4d-14c8-4574-9577-68f1006d5cb1" (UID: "82c3ed4d-14c8-4574-9577-68f1006d5cb1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:07:55.964179 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:55.964157    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82c3ed4d-14c8-4574-9577-68f1006d5cb1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "82c3ed4d-14c8-4574-9577-68f1006d5cb1" (UID: "82c3ed4d-14c8-4574-9577-68f1006d5cb1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:07:55.964356 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:55.964327    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82c3ed4d-14c8-4574-9577-68f1006d5cb1-kube-api-access-gc4j8" (OuterVolumeSpecName: "kube-api-access-gc4j8") pod "82c3ed4d-14c8-4574-9577-68f1006d5cb1" (UID: "82c3ed4d-14c8-4574-9577-68f1006d5cb1"). InnerVolumeSpecName "kube-api-access-gc4j8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:07:56.062927 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.062890    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82c3ed4d-14c8-4574-9577-68f1006d5cb1-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:07:56.062927 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.062919    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82c3ed4d-14c8-4574-9577-68f1006d5cb1-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:07:56.062927 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.062929    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc4j8\" (UniqueName: \"kubernetes.io/projected/82c3ed4d-14c8-4574-9577-68f1006d5cb1-kube-api-access-gc4j8\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:07:56.725725 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.725691    2577 generic.go:356] "Generic (PLEG): container finished" podID="82c3ed4d-14c8-4574-9577-68f1006d5cb1" containerID="8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261" exitCode=0
Jul 15 14:07:56.726153 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.725744    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"
Jul 15 14:07:56.726153 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.725770    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv" event={"ID":"82c3ed4d-14c8-4574-9577-68f1006d5cb1","Type":"ContainerDied","Data":"8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261"}
Jul 15 14:07:56.726153 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.725811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv" event={"ID":"82c3ed4d-14c8-4574-9577-68f1006d5cb1","Type":"ContainerDied","Data":"a0d84879a8212f7163382cc43dbe87978ac25a731f88a2899b2fe3b59eecdbbf"}
Jul 15 14:07:56.726153 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.725832    2577 scope.go:117] "RemoveContainer" containerID="8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261"
Jul 15 14:07:56.734425 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.734404    2577 scope.go:117] "RemoveContainer" containerID="8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261"
Jul 15 14:07:56.734780 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:07:56.734756    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261\": container with ID starting with 8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261 not found: ID does not exist" containerID="8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261"
Jul 15 14:07:56.734825 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.734792    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261"} err="failed to get container status \"8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261\": rpc error: code = NotFound desc = could not find container \"8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261\": container with ID starting with 8ee30d9e1fd53a641bf2f74ad273b769dc8db0b5e507afc369bc3c9c9d07d261 not found: ID does not exist"
Jul 15 14:07:56.755745 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.755711    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"]
Jul 15 14:07:56.763531 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:56.763505    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-16550d-5b45687cbb-ph9hv"]
Jul 15 14:07:58.146506 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:07:58.146475    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82c3ed4d-14c8-4574-9577-68f1006d5cb1" path="/var/lib/kubelet/pods/82c3ed4d-14c8-4574-9577-68f1006d5cb1/volumes"
Jul 15 14:08:00.301985 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:00.301951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:06.435550 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:06.435466    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:06.535921 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:06.535888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:11.028596 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.028554    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:11.304792 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.304693    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:11.423780 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.423749    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm"]
Jul 15 14:08:11.424217 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.424198    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82c3ed4d-14c8-4574-9577-68f1006d5cb1" containerName="maas-api"
Jul 15 14:08:11.424217 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.424218    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="82c3ed4d-14c8-4574-9577-68f1006d5cb1" containerName="maas-api"
Jul 15 14:08:11.424411 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.424249    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38412d9f-0a05-4a56-a6e4-e6a1c094e018" containerName="revoke-keys"
Jul 15 14:08:11.424411 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.424258    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="38412d9f-0a05-4a56-a6e4-e6a1c094e018" containerName="revoke-keys"
Jul 15 14:08:11.424411 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.424367    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="38412d9f-0a05-4a56-a6e4-e6a1c094e018" containerName="revoke-keys"
Jul 15 14:08:11.424411 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.424386    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="82c3ed4d-14c8-4574-9577-68f1006d5cb1" containerName="maas-api"
Jul 15 14:08:11.428891 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.428871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm"
Jul 15 14:08:11.431605 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.431586    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 14:08:11.436218 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.436194    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm"]
Jul 15 14:08:11.497188 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.497146    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e501a9fe-9768-4165-a24e-836decc5676d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm\" (UID: \"e501a9fe-9768-4165-a24e-836decc5676d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm"
Jul 15 14:08:11.598525 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.598438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e501a9fe-9768-4165-a24e-836decc5676d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm\" (UID: \"e501a9fe-9768-4165-a24e-836decc5676d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm"
Jul 15 14:08:11.599062 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.599040    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e501a9fe-9768-4165-a24e-836decc5676d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm\" (UID: \"e501a9fe-9768-4165-a24e-836decc5676d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm"
Jul 15 14:08:11.740201 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.740163    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm"
Jul 15 14:08:11.882996 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.882923    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm"]
Jul 15 14:08:11.886542 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:08:11.886513    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode501a9fe_9768_4165_a24e_836decc5676d.slice/crio-53d3ba6f986241b27f8d67c1bac0ea39ab381877512d5e800065b43a160b94d9 WatchSource:0}: Error finding container 53d3ba6f986241b27f8d67c1bac0ea39ab381877512d5e800065b43a160b94d9: Status 404 returned error can't find the container with id 53d3ba6f986241b27f8d67c1bac0ea39ab381877512d5e800065b43a160b94d9
Jul 15 14:08:11.960867 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:11.960836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:12.061441 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:12.061410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:12.804844 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:12.804811    2577 generic.go:356] "Generic (PLEG): container finished" podID="e501a9fe-9768-4165-a24e-836decc5676d" containerID="93d71c6cccfd249b959d3bf2e9104b7d2c5094fd0f6f1dc8a96ecde83f7085bc" exitCode=0
Jul 15 14:08:12.804997 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:12.804907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm" event={"ID":"e501a9fe-9768-4165-a24e-836decc5676d","Type":"ContainerDied","Data":"93d71c6cccfd249b959d3bf2e9104b7d2c5094fd0f6f1dc8a96ecde83f7085bc"}
Jul 15 14:08:12.805061 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:12.804996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm" event={"ID":"e501a9fe-9768-4165-a24e-836decc5676d","Type":"ContainerStarted","Data":"53d3ba6f986241b27f8d67c1bac0ea39ab381877512d5e800065b43a160b94d9"}
Jul 15 14:08:13.945075 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:13.945046    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm"
Jul 15 14:08:14.018487 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:14.018451    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e501a9fe-9768-4165-a24e-836decc5676d-maas-api-service-ca\") pod \"e501a9fe-9768-4165-a24e-836decc5676d\" (UID: \"e501a9fe-9768-4165-a24e-836decc5676d\") "
Jul 15 14:08:14.018828 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:14.018805    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e501a9fe-9768-4165-a24e-836decc5676d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e501a9fe-9768-4165-a24e-836decc5676d" (UID: "e501a9fe-9768-4165-a24e-836decc5676d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:08:14.119785 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:14.119701    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e501a9fe-9768-4165-a24e-836decc5676d-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:08:14.815955 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:14.815928    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm"
Jul 15 14:08:14.816099 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:14.815927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm" event={"ID":"e501a9fe-9768-4165-a24e-836decc5676d","Type":"ContainerDied","Data":"53d3ba6f986241b27f8d67c1bac0ea39ab381877512d5e800065b43a160b94d9"}
Jul 15 14:08:14.816099 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:14.816052    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53d3ba6f986241b27f8d67c1bac0ea39ab381877512d5e800065b43a160b94d9"
Jul 15 14:08:21.814946 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:21.814915    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"]
Jul 15 14:08:21.815410 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:21.815118    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt" podUID="09a6f8bf-ac02-466b-9a67-40191c591922" containerName="maas-api" containerID="cri-o://0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767" gracePeriod=30
Jul 15 14:08:21.840646 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:21.840606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:22.260494 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.260465    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:08:22.396021 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.395935    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09a6f8bf-ac02-466b-9a67-40191c591922-openshift-service-ca\") pod \"09a6f8bf-ac02-466b-9a67-40191c591922\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") "
Jul 15 14:08:22.396171 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.396035    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09a6f8bf-ac02-466b-9a67-40191c591922-maas-api-tls\") pod \"09a6f8bf-ac02-466b-9a67-40191c591922\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") "
Jul 15 14:08:22.396171 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.396087    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnn5p\" (UniqueName: \"kubernetes.io/projected/09a6f8bf-ac02-466b-9a67-40191c591922-kube-api-access-mnn5p\") pod \"09a6f8bf-ac02-466b-9a67-40191c591922\" (UID: \"09a6f8bf-ac02-466b-9a67-40191c591922\") "
Jul 15 14:08:22.396374 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.396348    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09a6f8bf-ac02-466b-9a67-40191c591922-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "09a6f8bf-ac02-466b-9a67-40191c591922" (UID: "09a6f8bf-ac02-466b-9a67-40191c591922"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:08:22.398311 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.398281    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09a6f8bf-ac02-466b-9a67-40191c591922-kube-api-access-mnn5p" (OuterVolumeSpecName: "kube-api-access-mnn5p") pod "09a6f8bf-ac02-466b-9a67-40191c591922" (UID: "09a6f8bf-ac02-466b-9a67-40191c591922"). InnerVolumeSpecName "kube-api-access-mnn5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:08:22.398426 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.398361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09a6f8bf-ac02-466b-9a67-40191c591922-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "09a6f8bf-ac02-466b-9a67-40191c591922" (UID: "09a6f8bf-ac02-466b-9a67-40191c591922"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:08:22.496828 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.496794    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09a6f8bf-ac02-466b-9a67-40191c591922-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:08:22.496828 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.496823    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnn5p\" (UniqueName: \"kubernetes.io/projected/09a6f8bf-ac02-466b-9a67-40191c591922-kube-api-access-mnn5p\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:08:22.496828 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.496833    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09a6f8bf-ac02-466b-9a67-40191c591922-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:08:22.854793 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.854757    2577 generic.go:356] "Generic (PLEG): container finished" podID="09a6f8bf-ac02-466b-9a67-40191c591922" containerID="0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767" exitCode=0
Jul 15 14:08:22.855228 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.854803    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"
Jul 15 14:08:22.855228 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.854855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt" event={"ID":"09a6f8bf-ac02-466b-9a67-40191c591922","Type":"ContainerDied","Data":"0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767"}
Jul 15 14:08:22.855228 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.854908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt" event={"ID":"09a6f8bf-ac02-466b-9a67-40191c591922","Type":"ContainerDied","Data":"dac1e06f5fbc40206a83ca0dd849a298724901002b6b048d6ab4b741f1cd7570"}
Jul 15 14:08:22.855228 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.854931    2577 scope.go:117] "RemoveContainer" containerID="0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767"
Jul 15 14:08:22.864136 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.864120    2577 scope.go:117] "RemoveContainer" containerID="0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767"
Jul 15 14:08:22.864476 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:08:22.864457    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767\": container with ID starting with 0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767 not found: ID does not exist" containerID="0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767"
Jul 15 14:08:22.864539 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.864485    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767"} err="failed to get container status \"0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767\": rpc error: code = NotFound desc = could not find container \"0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767\": container with ID starting with 0649434e0ada5d8e93a8bb050a832db972921453779662f2d702b28fc43cc767 not found: ID does not exist"
Jul 15 14:08:22.876316 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.876292    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"]
Jul 15 14:08:22.880446 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:22.880422    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-59923e-56c8df9997-vccdt"]
Jul 15 14:08:24.146947 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:24.146916    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09a6f8bf-ac02-466b-9a67-40191c591922" path="/var/lib/kubelet/pods/09a6f8bf-ac02-466b-9a67-40191c591922/volumes"
Jul 15 14:08:26.601316 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:26.601234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:32.928019 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:32.927981    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:33.065965 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:33.065926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:36.383398 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:36.383360    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:36.483784 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:36.483752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:37.241483 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:37.241442    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:37.492431 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:37.492326    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:37.868624 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:37.868588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:38.215456 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:38.215372    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:38.468916 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:38.468830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:41.409084 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.409055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:41.510959 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.510923    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"]
Jul 15 14:08:41.511362 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.511347    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09a6f8bf-ac02-466b-9a67-40191c591922" containerName="maas-api"
Jul 15 14:08:41.511414 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.511367    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09a6f8bf-ac02-466b-9a67-40191c591922" containerName="maas-api"
Jul 15 14:08:41.511414 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.511382    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e501a9fe-9768-4165-a24e-836decc5676d" containerName="revoke-keys"
Jul 15 14:08:41.511414 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.511389    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e501a9fe-9768-4165-a24e-836decc5676d" containerName="revoke-keys"
Jul 15 14:08:41.511512 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.511450    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e501a9fe-9768-4165-a24e-836decc5676d" containerName="revoke-keys"
Jul 15 14:08:41.511512 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.511459    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="09a6f8bf-ac02-466b-9a67-40191c591922" containerName="maas-api"
Jul 15 14:08:41.513077 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.512992    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:41.514609 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.514586    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.517482 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.517462    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-69a378-serving-cert\""
Jul 15 14:08:41.525745 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.525705    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"]
Jul 15 14:08:41.560088 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.560056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzdt9\" (UniqueName: \"kubernetes.io/projected/7fcf594e-bc29-4950-b2af-be84b7132305-kube-api-access-zzdt9\") pod \"maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.560088 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.560101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fcf594e-bc29-4950-b2af-be84b7132305-maas-api-tls\") pod \"maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.560379 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.560138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fcf594e-bc29-4950-b2af-be84b7132305-openshift-service-ca\") pod \"maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.661451 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.661358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fcf594e-bc29-4950-b2af-be84b7132305-openshift-service-ca\") pod \"maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.661451 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.661427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzdt9\" (UniqueName: \"kubernetes.io/projected/7fcf594e-bc29-4950-b2af-be84b7132305-kube-api-access-zzdt9\") pod \"maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.661640 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.661456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fcf594e-bc29-4950-b2af-be84b7132305-maas-api-tls\") pod \"maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.662059 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.662040    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fcf594e-bc29-4950-b2af-be84b7132305-openshift-service-ca\") pod \"maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.663947 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.663930    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fcf594e-bc29-4950-b2af-be84b7132305-maas-api-tls\") pod \"maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.670599 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.670560    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzdt9\" (UniqueName: \"kubernetes.io/projected/7fcf594e-bc29-4950-b2af-be84b7132305-kube-api-access-zzdt9\") pod \"maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.828224 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.828183    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:41.994535 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.994507    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"]
Jul 15 14:08:41.997122 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:08:41.997093    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fcf594e_bc29_4950_b2af_be84b7132305.slice/crio-72f97cbe8c73e91c00b87d05a9bb1bec4afcf295fab84ee5d8fc1cb3e70d39cd WatchSource:0}: Error finding container 72f97cbe8c73e91c00b87d05a9bb1bec4afcf295fab84ee5d8fc1cb3e70d39cd: Status 404 returned error can't find the container with id 72f97cbe8c73e91c00b87d05a9bb1bec4afcf295fab84ee5d8fc1cb3e70d39cd
Jul 15 14:08:41.998447 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:41.998427    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 14:08:42.953953 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:42.953848    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx" event={"ID":"7fcf594e-bc29-4950-b2af-be84b7132305","Type":"ContainerStarted","Data":"52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e"}
Jul 15 14:08:42.953953 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:42.953902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx" event={"ID":"7fcf594e-bc29-4950-b2af-be84b7132305","Type":"ContainerStarted","Data":"72f97cbe8c73e91c00b87d05a9bb1bec4afcf295fab84ee5d8fc1cb3e70d39cd"}
Jul 15 14:08:42.954393 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:42.954054    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:42.976590 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:42.976537    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx" podStartSLOduration=1.458099717 podStartE2EDuration="1.976520213s" totalImagesPullingTime="518.420496ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:08:42.97551551 +0000 UTC m=+2367.318450590" watchObservedRunningTime="2026-07-15 14:08:42.976520213 +0000 UTC m=+2367.319455292"
Jul 15 14:08:43.488191 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:43.488161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:43.574453 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:43.574420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:46.000419 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:46.000389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:46.471855 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:46.471819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:46.826544 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:46.826514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:47.220959 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:47.220882    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:48.965390 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:48.965348    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:08:49.593294 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.593244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:49.693845 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.693810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:49.700560 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.700532    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"]
Jul 15 14:08:49.704335 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.704314    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:49.708116 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.708088    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f79741-serving-cert\""
Jul 15 14:08:49.725291 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.725247    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"]
Jul 15 14:08:49.736924 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.736897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84d487-a2bb-49be-9a5d-3563908200f2-maas-api-tls\") pod \"maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:49.737064 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.736945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfxvz\" (UniqueName: \"kubernetes.io/projected/0c84d487-a2bb-49be-9a5d-3563908200f2-kube-api-access-nfxvz\") pod \"maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:49.737109 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.737081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c84d487-a2bb-49be-9a5d-3563908200f2-openshift-service-ca\") pod \"maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:49.781605 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.781509    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:49.837962 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.837923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84d487-a2bb-49be-9a5d-3563908200f2-maas-api-tls\") pod \"maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:49.838133 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.837974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfxvz\" (UniqueName: \"kubernetes.io/projected/0c84d487-a2bb-49be-9a5d-3563908200f2-kube-api-access-nfxvz\") pod \"maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:49.838133 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.838029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c84d487-a2bb-49be-9a5d-3563908200f2-openshift-service-ca\") pod \"maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:49.838133 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:08:49.838078    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-serving-cert: secret "maas-api-e2e-api-b-f79741-serving-cert" not found
Jul 15 14:08:49.838364 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:08:49.838148    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c84d487-a2bb-49be-9a5d-3563908200f2-maas-api-tls podName:0c84d487-a2bb-49be-9a5d-3563908200f2 nodeName:}" failed. No retries permitted until 2026-07-15 14:08:50.338130481 +0000 UTC m=+2374.681065542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0c84d487-a2bb-49be-9a5d-3563908200f2-maas-api-tls") pod "maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4" (UID: "0c84d487-a2bb-49be-9a5d-3563908200f2") : secret "maas-api-e2e-api-b-f79741-serving-cert" not found
Jul 15 14:08:49.838717 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.838692    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c84d487-a2bb-49be-9a5d-3563908200f2-openshift-service-ca\") pod \"maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:49.848209 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:49.848149    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfxvz\" (UniqueName: \"kubernetes.io/projected/0c84d487-a2bb-49be-9a5d-3563908200f2-kube-api-access-nfxvz\") pod \"maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:50.100346 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:50.100224    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:50.342022 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:50.341983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84d487-a2bb-49be-9a5d-3563908200f2-maas-api-tls\") pod \"maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:50.344653 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:50.344624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84d487-a2bb-49be-9a5d-3563908200f2-maas-api-tls\") pod \"maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:50.443864 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:50.443770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:50.543777 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:50.543742    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:50.614805 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:50.614767    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:50.768671 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:50.768623    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"]
Jul 15 14:08:50.768874 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:08:50.768851    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c84d487_a2bb_49be_9a5d_3563908200f2.slice/crio-e5658b9b4d0db6d2ad7b9d5b3442e5831bbc8fdc8bfdc3f635528a9fe0d22968 WatchSource:0}: Error finding container e5658b9b4d0db6d2ad7b9d5b3442e5831bbc8fdc8bfdc3f635528a9fe0d22968: Status 404 returned error can't find the container with id e5658b9b4d0db6d2ad7b9d5b3442e5831bbc8fdc8bfdc3f635528a9fe0d22968
Jul 15 14:08:50.996170 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:50.996077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4" event={"ID":"0c84d487-a2bb-49be-9a5d-3563908200f2","Type":"ContainerStarted","Data":"e5658b9b4d0db6d2ad7b9d5b3442e5831bbc8fdc8bfdc3f635528a9fe0d22968"}
Jul 15 14:08:52.002149 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:52.002111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4" event={"ID":"0c84d487-a2bb-49be-9a5d-3563908200f2","Type":"ContainerStarted","Data":"3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14"}
Jul 15 14:08:52.002629 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:52.002248    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:08:52.025574 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:52.025520    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4" podStartSLOduration=2.634494984 podStartE2EDuration="3.025503756s" totalImagesPullingTime="391.008772ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:08:52.024208307 +0000 UTC m=+2376.367143411" watchObservedRunningTime="2026-07-15 14:08:52.025503756 +0000 UTC m=+2376.368438836"
Jul 15 14:08:54.159935 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:54.159900    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:56.050724 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:56.050694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:08:58.013636 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:08:58.013609    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:09:02.632689 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:02.632654    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx"]
Jul 15 14:09:02.642700 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:02.642668    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx"
Jul 15 14:09:02.644210 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:02.644179    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx"]
Jul 15 14:09:02.645401 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:02.645309    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 14:09:02.746419 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:02.746384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc3ae06-9408-44ca-b81c-81517290bbc5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-69a378-nzbjx\" (UID: \"2fc3ae06-9408-44ca-b81c-81517290bbc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx"
Jul 15 14:09:02.847683 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:02.847645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc3ae06-9408-44ca-b81c-81517290bbc5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-69a378-nzbjx\" (UID: \"2fc3ae06-9408-44ca-b81c-81517290bbc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx"
Jul 15 14:09:02.848249 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:02.848229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc3ae06-9408-44ca-b81c-81517290bbc5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-69a378-nzbjx\" (UID: \"2fc3ae06-9408-44ca-b81c-81517290bbc5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx"
Jul 15 14:09:02.956337 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:02.956219    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx"
Jul 15 14:09:03.108478 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:03.108450    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx"]
Jul 15 14:09:03.110848 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:09:03.110819    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc3ae06_9408_44ca_b81c_81517290bbc5.slice/crio-a3596c93a10ddff15f78c0cc5ebbc27e9029af7c51b5a92afa43fae94a962384 WatchSource:0}: Error finding container a3596c93a10ddff15f78c0cc5ebbc27e9029af7c51b5a92afa43fae94a962384: Status 404 returned error can't find the container with id a3596c93a10ddff15f78c0cc5ebbc27e9029af7c51b5a92afa43fae94a962384
Jul 15 14:09:03.181160 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:03.181130    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:03.303850 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:03.303813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:04.066963 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:04.066934    2577 generic.go:356] "Generic (PLEG): container finished" podID="2fc3ae06-9408-44ca-b81c-81517290bbc5" containerID="7609aa24b5f541d52edf4ef878e170b9d002802952cfdf74af020fed694c0027" exitCode=0
Jul 15 14:09:04.067418 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:04.067003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx" event={"ID":"2fc3ae06-9408-44ca-b81c-81517290bbc5","Type":"ContainerDied","Data":"7609aa24b5f541d52edf4ef878e170b9d002802952cfdf74af020fed694c0027"}
Jul 15 14:09:04.067418 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:04.067046    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx" event={"ID":"2fc3ae06-9408-44ca-b81c-81517290bbc5","Type":"ContainerStarted","Data":"a3596c93a10ddff15f78c0cc5ebbc27e9029af7c51b5a92afa43fae94a962384"}
Jul 15 14:09:05.208809 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:05.208785    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx"
Jul 15 14:09:05.269088 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:05.269051    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc3ae06-9408-44ca-b81c-81517290bbc5-maas-api-service-ca\") pod \"2fc3ae06-9408-44ca-b81c-81517290bbc5\" (UID: \"2fc3ae06-9408-44ca-b81c-81517290bbc5\") "
Jul 15 14:09:05.269443 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:05.269421    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fc3ae06-9408-44ca-b81c-81517290bbc5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2fc3ae06-9408-44ca-b81c-81517290bbc5" (UID: "2fc3ae06-9408-44ca-b81c-81517290bbc5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:09:05.369847 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:05.369754    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fc3ae06-9408-44ca-b81c-81517290bbc5-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:09:06.078145 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:06.078116    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx"
Jul 15 14:09:06.078145 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:06.078128    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-69a378-nzbjx" event={"ID":"2fc3ae06-9408-44ca-b81c-81517290bbc5","Type":"ContainerDied","Data":"a3596c93a10ddff15f78c0cc5ebbc27e9029af7c51b5a92afa43fae94a962384"}
Jul 15 14:09:06.078380 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:06.078176    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3596c93a10ddff15f78c0cc5ebbc27e9029af7c51b5a92afa43fae94a962384"
Jul 15 14:09:07.122315 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:07.122249    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:11.381852 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:11.381814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:11.486220 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:11.486186    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:13.227906 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.227871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:13.325085 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.325055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:13.468796 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.468759    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"]
Jul 15 14:09:13.469028 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.469006    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx" podUID="7fcf594e-bc29-4950-b2af-be84b7132305" containerName="maas-api" containerID="cri-o://52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e" gracePeriod=30
Jul 15 14:09:13.505232 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.505198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:13.722108 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.722087    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:09:13.846877 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.846838    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fcf594e-bc29-4950-b2af-be84b7132305-maas-api-tls\") pod \"7fcf594e-bc29-4950-b2af-be84b7132305\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") "
Jul 15 14:09:13.847088 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.846886    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzdt9\" (UniqueName: \"kubernetes.io/projected/7fcf594e-bc29-4950-b2af-be84b7132305-kube-api-access-zzdt9\") pod \"7fcf594e-bc29-4950-b2af-be84b7132305\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") "
Jul 15 14:09:13.847088 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.846915    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fcf594e-bc29-4950-b2af-be84b7132305-openshift-service-ca\") pod \"7fcf594e-bc29-4950-b2af-be84b7132305\" (UID: \"7fcf594e-bc29-4950-b2af-be84b7132305\") "
Jul 15 14:09:13.847373 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.847351    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fcf594e-bc29-4950-b2af-be84b7132305-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7fcf594e-bc29-4950-b2af-be84b7132305" (UID: "7fcf594e-bc29-4950-b2af-be84b7132305"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:09:13.849199 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.849176    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fcf594e-bc29-4950-b2af-be84b7132305-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7fcf594e-bc29-4950-b2af-be84b7132305" (UID: "7fcf594e-bc29-4950-b2af-be84b7132305"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:09:13.849256 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.849204    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fcf594e-bc29-4950-b2af-be84b7132305-kube-api-access-zzdt9" (OuterVolumeSpecName: "kube-api-access-zzdt9") pod "7fcf594e-bc29-4950-b2af-be84b7132305" (UID: "7fcf594e-bc29-4950-b2af-be84b7132305"). InnerVolumeSpecName "kube-api-access-zzdt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:09:13.948484 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.948437    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzdt9\" (UniqueName: \"kubernetes.io/projected/7fcf594e-bc29-4950-b2af-be84b7132305-kube-api-access-zzdt9\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:09:13.948484 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.948477    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fcf594e-bc29-4950-b2af-be84b7132305-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:09:13.948484 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:13.948491    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fcf594e-bc29-4950-b2af-be84b7132305-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:09:14.118041 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:14.117955    2577 generic.go:356] "Generic (PLEG): container finished" podID="7fcf594e-bc29-4950-b2af-be84b7132305" containerID="52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e" exitCode=0
Jul 15 14:09:14.118041 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:14.118000    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"
Jul 15 14:09:14.118217 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:14.118038    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx" event={"ID":"7fcf594e-bc29-4950-b2af-be84b7132305","Type":"ContainerDied","Data":"52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e"}
Jul 15 14:09:14.118217 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:14.118095    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx" event={"ID":"7fcf594e-bc29-4950-b2af-be84b7132305","Type":"ContainerDied","Data":"72f97cbe8c73e91c00b87d05a9bb1bec4afcf295fab84ee5d8fc1cb3e70d39cd"}
Jul 15 14:09:14.118217 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:14.118118    2577 scope.go:117] "RemoveContainer" containerID="52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e"
Jul 15 14:09:14.127515 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:14.127497    2577 scope.go:117] "RemoveContainer" containerID="52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e"
Jul 15 14:09:14.127772 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:09:14.127751    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e\": container with ID starting with 52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e not found: ID does not exist" containerID="52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e"
Jul 15 14:09:14.127842 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:14.127781    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e"} err="failed to get container status \"52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e\": rpc error: code = NotFound desc = could not find container \"52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e\": container with ID starting with 52860982f23fb305d86d4a972fa1e610964cc11c5fd38da03e7f5b1e2626055e not found: ID does not exist"
Jul 15 14:09:14.147175 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:14.147144    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"]
Jul 15 14:09:14.147779 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:14.147760    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-69a378-55c5d7f8b4-vvnxx"]
Jul 15 14:09:16.147195 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:16.147162    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fcf594e-bc29-4950-b2af-be84b7132305" path="/var/lib/kubelet/pods/7fcf594e-bc29-4950-b2af-be84b7132305/volumes"
Jul 15 14:09:16.233756 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:16.233727    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 14:09:16.240018 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:16.239992    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 14:09:17.732511 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:17.732482    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:23.870997 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:23.870962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:23.933737 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:23.933699    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:28.502666 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.502629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:28.762912 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.762880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:28.889552 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.889517    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn"]
Jul 15 14:09:28.889868 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.889856    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fc3ae06-9408-44ca-b81c-81517290bbc5" containerName="revoke-keys"
Jul 15 14:09:28.889912 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.889870    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fc3ae06-9408-44ca-b81c-81517290bbc5" containerName="revoke-keys"
Jul 15 14:09:28.889912 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.889891    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fcf594e-bc29-4950-b2af-be84b7132305" containerName="maas-api"
Jul 15 14:09:28.889912 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.889901    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fcf594e-bc29-4950-b2af-be84b7132305" containerName="maas-api"
Jul 15 14:09:28.890008 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.889952    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fc3ae06-9408-44ca-b81c-81517290bbc5" containerName="revoke-keys"
Jul 15 14:09:28.890008 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.889964    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fcf594e-bc29-4950-b2af-be84b7132305" containerName="maas-api"
Jul 15 14:09:28.893375 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.893354    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn"
Jul 15 14:09:28.896905 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.896883    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 14:09:28.902464 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.902382    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn"]
Jul 15 14:09:28.978519 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:28.978482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01804600-9d29-4036-9a77-2083eb7bbbcc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f79741-xtwwn\" (UID: \"01804600-9d29-4036-9a77-2083eb7bbbcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn"
Jul 15 14:09:29.079027 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:29.078943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01804600-9d29-4036-9a77-2083eb7bbbcc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f79741-xtwwn\" (UID: \"01804600-9d29-4036-9a77-2083eb7bbbcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn"
Jul 15 14:09:29.079600 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:29.079581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01804600-9d29-4036-9a77-2083eb7bbbcc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f79741-xtwwn\" (UID: \"01804600-9d29-4036-9a77-2083eb7bbbcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn"
Jul 15 14:09:29.206536 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:29.206503    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn"
Jul 15 14:09:29.352053 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:29.351965    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn"]
Jul 15 14:09:29.355077 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:09:29.355049    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01804600_9d29_4036_9a77_2083eb7bbbcc.slice/crio-5b8aaef2f5c3b9e5255cf22b567dbf874149eaa1050eccee42e3bb0345783ed8 WatchSource:0}: Error finding container 5b8aaef2f5c3b9e5255cf22b567dbf874149eaa1050eccee42e3bb0345783ed8: Status 404 returned error can't find the container with id 5b8aaef2f5c3b9e5255cf22b567dbf874149eaa1050eccee42e3bb0345783ed8
Jul 15 14:09:29.466562 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:29.466504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:29.571616 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:29.571585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:30.197466 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:30.197427    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn" event={"ID":"01804600-9d29-4036-9a77-2083eb7bbbcc","Type":"ContainerStarted","Data":"4a838061ba3cf988c78feb68c939b29a94ea17b25145ec000ce267316bf430ee"}
Jul 15 14:09:30.197650 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:30.197493    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn" event={"ID":"01804600-9d29-4036-9a77-2083eb7bbbcc","Type":"ContainerStarted","Data":"5b8aaef2f5c3b9e5255cf22b567dbf874149eaa1050eccee42e3bb0345783ed8"}
Jul 15 14:09:30.219069 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:30.219016    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn" podStartSLOduration=2.219000938 podStartE2EDuration="2.219000938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:09:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:09:30.217448277 +0000 UTC m=+2414.560383357" watchObservedRunningTime="2026-07-15 14:09:30.219000938 +0000 UTC m=+2414.561936043"
Jul 15 14:09:31.203900 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:31.203866    2577 generic.go:356] "Generic (PLEG): container finished" podID="01804600-9d29-4036-9a77-2083eb7bbbcc" containerID="4a838061ba3cf988c78feb68c939b29a94ea17b25145ec000ce267316bf430ee" exitCode=0
Jul 15 14:09:31.204311 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:31.203950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn" event={"ID":"01804600-9d29-4036-9a77-2083eb7bbbcc","Type":"ContainerDied","Data":"4a838061ba3cf988c78feb68c939b29a94ea17b25145ec000ce267316bf430ee"}
Jul 15 14:09:32.339726 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:32.339698    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn"
Jul 15 14:09:32.407062 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:32.407028    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01804600-9d29-4036-9a77-2083eb7bbbcc-maas-api-service-ca\") pod \"01804600-9d29-4036-9a77-2083eb7bbbcc\" (UID: \"01804600-9d29-4036-9a77-2083eb7bbbcc\") "
Jul 15 14:09:32.407427 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:32.407403    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01804600-9d29-4036-9a77-2083eb7bbbcc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01804600-9d29-4036-9a77-2083eb7bbbcc" (UID: "01804600-9d29-4036-9a77-2083eb7bbbcc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:09:32.508381 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:32.508353    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01804600-9d29-4036-9a77-2083eb7bbbcc-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:09:33.214543 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:33.214502    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn" event={"ID":"01804600-9d29-4036-9a77-2083eb7bbbcc","Type":"ContainerDied","Data":"5b8aaef2f5c3b9e5255cf22b567dbf874149eaa1050eccee42e3bb0345783ed8"}
Jul 15 14:09:33.214708 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:33.214547    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f79741-xtwwn"
Jul 15 14:09:33.214708 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:33.214564    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b8aaef2f5c3b9e5255cf22b567dbf874149eaa1050eccee42e3bb0345783ed8"
Jul 15 14:09:39.439380 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.439304    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"]
Jul 15 14:09:39.439767 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.439495    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4" podUID="0c84d487-a2bb-49be-9a5d-3563908200f2" containerName="maas-api" containerID="cri-o://3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14" gracePeriod=30
Jul 15 14:09:39.470597 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.470560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:39.695154 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.695092    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:09:39.770512 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.770471    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfxvz\" (UniqueName: \"kubernetes.io/projected/0c84d487-a2bb-49be-9a5d-3563908200f2-kube-api-access-nfxvz\") pod \"0c84d487-a2bb-49be-9a5d-3563908200f2\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") "
Jul 15 14:09:39.770512 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.770512    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c84d487-a2bb-49be-9a5d-3563908200f2-openshift-service-ca\") pod \"0c84d487-a2bb-49be-9a5d-3563908200f2\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") "
Jul 15 14:09:39.770737 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.770679    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84d487-a2bb-49be-9a5d-3563908200f2-maas-api-tls\") pod \"0c84d487-a2bb-49be-9a5d-3563908200f2\" (UID: \"0c84d487-a2bb-49be-9a5d-3563908200f2\") "
Jul 15 14:09:39.770826 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.770809    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c84d487-a2bb-49be-9a5d-3563908200f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0c84d487-a2bb-49be-9a5d-3563908200f2" (UID: "0c84d487-a2bb-49be-9a5d-3563908200f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:09:39.770966 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.770945    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c84d487-a2bb-49be-9a5d-3563908200f2-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:09:39.772784 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.772750    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c84d487-a2bb-49be-9a5d-3563908200f2-kube-api-access-nfxvz" (OuterVolumeSpecName: "kube-api-access-nfxvz") pod "0c84d487-a2bb-49be-9a5d-3563908200f2" (UID: "0c84d487-a2bb-49be-9a5d-3563908200f2"). InnerVolumeSpecName "kube-api-access-nfxvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:09:39.772889 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.772781    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c84d487-a2bb-49be-9a5d-3563908200f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0c84d487-a2bb-49be-9a5d-3563908200f2" (UID: "0c84d487-a2bb-49be-9a5d-3563908200f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:09:39.871702 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.871668    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c84d487-a2bb-49be-9a5d-3563908200f2-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:09:39.871702 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:39.871697    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfxvz\" (UniqueName: \"kubernetes.io/projected/0c84d487-a2bb-49be-9a5d-3563908200f2-kube-api-access-nfxvz\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:09:40.251230 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:40.251194    2577 generic.go:356] "Generic (PLEG): container finished" podID="0c84d487-a2bb-49be-9a5d-3563908200f2" containerID="3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14" exitCode=0
Jul 15 14:09:40.251442 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:40.251244    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"
Jul 15 14:09:40.251442 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:40.251287    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4" event={"ID":"0c84d487-a2bb-49be-9a5d-3563908200f2","Type":"ContainerDied","Data":"3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14"}
Jul 15 14:09:40.251442 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:40.251344    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4" event={"ID":"0c84d487-a2bb-49be-9a5d-3563908200f2","Type":"ContainerDied","Data":"e5658b9b4d0db6d2ad7b9d5b3442e5831bbc8fdc8bfdc3f635528a9fe0d22968"}
Jul 15 14:09:40.251442 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:40.251371    2577 scope.go:117] "RemoveContainer" containerID="3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14"
Jul 15 14:09:40.260599 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:40.260582    2577 scope.go:117] "RemoveContainer" containerID="3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14"
Jul 15 14:09:40.260879 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:09:40.260852    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14\": container with ID starting with 3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14 not found: ID does not exist" containerID="3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14"
Jul 15 14:09:40.260971 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:40.260888    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14"} err="failed to get container status \"3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14\": rpc error: code = NotFound desc = could not find container \"3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14\": container with ID starting with 3735803cb6a9b6efb32a5db9f6733776bd829e43d63eaccb82c8a26911ad2d14 not found: ID does not exist"
Jul 15 14:09:40.275049 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:40.275024    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"]
Jul 15 14:09:40.284342 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:40.284319    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f79741-765b7bb47c-4cvk4"]
Jul 15 14:09:42.147164 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:42.147130    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c84d487-a2bb-49be-9a5d-3563908200f2" path="/var/lib/kubelet/pods/0c84d487-a2bb-49be-9a5d-3563908200f2/volumes"
Jul 15 14:09:43.994834 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:43.994796    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:50.647955 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:50.647910    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:50.748676 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:50.748637    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:54.768019 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:54.767979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:55.046110 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:55.046022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:55.443213 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:55.443122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:55.813101 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:55.813072    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:56.000034 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:55.999999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:58.790892 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:58.790850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:58.890548 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:58.890502    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:59.063739 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.063661    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"]
Jul 15 14:09:59.064014 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.064001    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c84d487-a2bb-49be-9a5d-3563908200f2" containerName="maas-api"
Jul 15 14:09:59.064064 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.064016    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c84d487-a2bb-49be-9a5d-3563908200f2" containerName="maas-api"
Jul 15 14:09:59.064064 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.064027    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01804600-9d29-4036-9a77-2083eb7bbbcc" containerName="revoke-keys"
Jul 15 14:09:59.064064 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.064032    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="01804600-9d29-4036-9a77-2083eb7bbbcc" containerName="revoke-keys"
Jul 15 14:09:59.064154 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.064096    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c84d487-a2bb-49be-9a5d-3563908200f2" containerName="maas-api"
Jul 15 14:09:59.064154 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.064107    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="01804600-9d29-4036-9a77-2083eb7bbbcc" containerName="revoke-keys"
Jul 15 14:09:59.067380 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.067359    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.069951 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.069930    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-2723e9-serving-cert\""
Jul 15 14:09:59.084459 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.084435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"]
Jul 15 14:09:59.101135 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.101111    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:59.136982 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.136942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p8qf\" (UniqueName: \"kubernetes.io/projected/c91fbced-410f-4545-ae70-633bcef9b5d9-kube-api-access-8p8qf\") pod \"maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.136982 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.136984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c91fbced-410f-4545-ae70-633bcef9b5d9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.137211 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.137069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c91fbced-410f-4545-ae70-633bcef9b5d9-maas-api-tls\") pod \"maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.188309 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.188254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:59.238208 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.238170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p8qf\" (UniqueName: \"kubernetes.io/projected/c91fbced-410f-4545-ae70-633bcef9b5d9-kube-api-access-8p8qf\") pod \"maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.238208 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.238206    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c91fbced-410f-4545-ae70-633bcef9b5d9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.238490 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.238296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c91fbced-410f-4545-ae70-633bcef9b5d9-maas-api-tls\") pod \"maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.238931 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.238910    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c91fbced-410f-4545-ae70-633bcef9b5d9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.240848 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.240828    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c91fbced-410f-4545-ae70-633bcef9b5d9-maas-api-tls\") pod \"maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.248949 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.248925    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p8qf\" (UniqueName: \"kubernetes.io/projected/c91fbced-410f-4545-ae70-633bcef9b5d9-kube-api-access-8p8qf\") pod \"maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.377783 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.377687    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:09:59.534033 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.533997    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"]
Jul 15 14:09:59.537713 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:09:59.537679    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91fbced_410f_4545_ae70_633bcef9b5d9.slice/crio-87ddfaac35a6639b7d98d90bb1db4915038634326e0151454d97189e1f98db7e WatchSource:0}: Error finding container 87ddfaac35a6639b7d98d90bb1db4915038634326e0151454d97189e1f98db7e: Status 404 returned error can't find the container with id 87ddfaac35a6639b7d98d90bb1db4915038634326e0151454d97189e1f98db7e
Jul 15 14:09:59.613730 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.613700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:09:59.744697 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:09:59.744606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:00.353219 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:00.353130    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm" event={"ID":"c91fbced-410f-4545-ae70-633bcef9b5d9","Type":"ContainerStarted","Data":"06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497"}
Jul 15 14:10:00.353219 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:00.353196    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:10:00.353739 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:00.353222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm" event={"ID":"c91fbced-410f-4545-ae70-633bcef9b5d9","Type":"ContainerStarted","Data":"87ddfaac35a6639b7d98d90bb1db4915038634326e0151454d97189e1f98db7e"}
Jul 15 14:10:03.359805 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:03.359770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:03.734580 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:03.734501    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:04.073334 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:04.073302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:04.175821 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:04.175788    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:05.628937 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:05.628902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:05.728927 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:05.728899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:06.363816 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:06.363791    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:10:06.384600 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:06.384548    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm" podStartSLOduration=6.918158098 podStartE2EDuration="7.384533827s" totalImagesPullingTime="466.375729ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:10:00.37896972 +0000 UTC m=+2444.721904822" watchObservedRunningTime="2026-07-15 14:10:06.384533827 +0000 UTC m=+2450.727468907"
Jul 15 14:10:07.147556 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.147505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:07.206889 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.206853    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"]
Jul 15 14:10:07.211872 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.211847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:07.215731 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.215706    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-bfdd40-serving-cert\""
Jul 15 14:10:07.220207 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.220179    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"]
Jul 15 14:10:07.245923 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.245886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:07.312704 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.312665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:07.312897 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.312751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn7nk\" (UniqueName: \"kubernetes.io/projected/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-kube-api-access-bn7nk\") pod \"maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:07.312897 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.312774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-maas-api-tls\") pod \"maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:07.344863 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.344832    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:07.413748 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.413648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:07.413934 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.413746    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn7nk\" (UniqueName: \"kubernetes.io/projected/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-kube-api-access-bn7nk\") pod \"maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:07.413934 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.413777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-maas-api-tls\") pod \"maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:07.413934 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:10:07.413899    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-serving-cert: secret "maas-api-e2e-shared-b-bfdd40-serving-cert" not found
Jul 15 14:10:07.414125 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:10:07.413969    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-maas-api-tls podName:d7460b09-79ea-4ef0-b394-2dcf72f00ee8 nodeName:}" failed. No retries permitted until 2026-07-15 14:10:07.913947958 +0000 UTC m=+2452.256883018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-maas-api-tls") pod "maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp" (UID: "d7460b09-79ea-4ef0-b394-2dcf72f00ee8") : secret "maas-api-e2e-shared-b-bfdd40-serving-cert" not found
Jul 15 14:10:07.414321 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.414302    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:07.424052 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.424024    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn7nk\" (UniqueName: \"kubernetes.io/projected/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-kube-api-access-bn7nk\") pod \"maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:07.805021 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.804987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:07.904235 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.904202    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:07.918508 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.918476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-maas-api-tls\") pod \"maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:07.921036 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:07.921011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-maas-api-tls\") pod \"maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:08.004896 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:08.004851    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:08.127786 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:08.127680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:08.270754 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:08.270729    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"]
Jul 15 14:10:08.393312 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:08.393206    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp" event={"ID":"d7460b09-79ea-4ef0-b394-2dcf72f00ee8","Type":"ContainerStarted","Data":"40c96be1b3d32a0e2c352f82bd2ab491ea42a452a42dc09d285a09b11c3d1a6c"}
Jul 15 14:10:09.399608 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:09.399566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp" event={"ID":"d7460b09-79ea-4ef0-b394-2dcf72f00ee8","Type":"ContainerStarted","Data":"eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b"}
Jul 15 14:10:09.400093 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:09.399708    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:09.422062 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:09.422008    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp" podStartSLOduration=2.080361595 podStartE2EDuration="2.421991831s" totalImagesPullingTime="341.630236ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:10:09.419349334 +0000 UTC m=+2453.762284426" watchObservedRunningTime="2026-07-15 14:10:09.421991831 +0000 UTC m=+2453.764926911"
Jul 15 14:10:11.623467 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:11.623438    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:15.409982 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:15.409950    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:10:20.426770 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:20.426727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:20.520352 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:20.520310    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:28.739705 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:28.739667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:28.842488 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:28.842448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:32.505050 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:32.505010    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:41.575444 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:41.575411    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:10:52.918632 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:10:52.918594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:11:11.907207 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:11:11.907173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:13:09.140752 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:13:09.140715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:13:10.547226 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:13:10.547191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:13:16.207253 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:13:16.207216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:13:17.889312 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:13:17.889276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:13:20.655459 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:13:20.655422    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:13:37.239332 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:13:37.239297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:13:37.342052 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:13:37.342017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:14:13.692509 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:14:13.692417    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:14:16.262658 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:14:16.262624    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 14:14:16.269193 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:14:16.269169    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jvcml_8062b1d7-6402-4374-ae45-de07b61247a3/ovn-acl-logging/0.log"
Jul 15 14:15:00.137560 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.137519    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr"]
Jul 15 14:15:00.141213 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.141183    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr"
Jul 15 14:15:00.144135 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.144113    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 14:15:00.147918 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.147862    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735415-rp82r"]
Jul 15 14:15:00.151691 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.151662    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-rp82r"
Jul 15 14:15:00.152121 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.152098    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78"]
Jul 15 14:15:00.153967 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.153948    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr"
Jul 15 14:15:00.156300 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.156216    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78"
Jul 15 14:15:00.157057 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.156121    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735415-rp82r"]
Jul 15 14:15:00.158834 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.157515    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr"]
Jul 15 14:15:00.161060 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.161029    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78"]
Jul 15 14:15:00.184877 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.184849    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-rp82r"
Jul 15 14:15:00.193083 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.193059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78"
Jul 15 14:15:00.342167 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.342144    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr"]
Jul 15 14:15:00.344049 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:15:00.344018    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48be81c0_979b_434b_a155_dd7f12bb973f.slice/crio-57609115354bdb964f2c4ec14b90cf42a3f7e063c8c091b0376fba4dc4b5f14f WatchSource:0}: Error finding container 57609115354bdb964f2c4ec14b90cf42a3f7e063c8c091b0376fba4dc4b5f14f: Status 404 returned error can't find the container with id 57609115354bdb964f2c4ec14b90cf42a3f7e063c8c091b0376fba4dc4b5f14f
Jul 15 14:15:00.580911 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:15:00.580864    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39bd64e8_37c7_4ae0_89bb_c5e255752efc.slice/crio-8300e9676f1cdbb56b2678610c57eb9fbba07c316aaa31d5dea483f2925c2a0b WatchSource:0}: Error finding container 8300e9676f1cdbb56b2678610c57eb9fbba07c316aaa31d5dea483f2925c2a0b: Status 404 returned error can't find the container with id 8300e9676f1cdbb56b2678610c57eb9fbba07c316aaa31d5dea483f2925c2a0b
Jul 15 14:15:00.581607 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:15:00.581535    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13dffdf_5156_4015_b86f_7c5c54d7e089.slice/crio-c521f8f49efb78956022f45de7ee5ba3ba494e581f72c2144d33fdee16b35539 WatchSource:0}: Error finding container c521f8f49efb78956022f45de7ee5ba3ba494e581f72c2144d33fdee16b35539: Status 404 returned error can't find the container with id c521f8f49efb78956022f45de7ee5ba3ba494e581f72c2144d33fdee16b35539
Jul 15 14:15:00.583014 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.582968    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78"]
Jul 15 14:15:00.585243 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.585222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735415-rp82r"]
Jul 15 14:15:00.858404 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.858369    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78" event={"ID":"39bd64e8-37c7-4ae0-89bb-c5e255752efc","Type":"ContainerStarted","Data":"8300e9676f1cdbb56b2678610c57eb9fbba07c316aaa31d5dea483f2925c2a0b"}
Jul 15 14:15:00.860098 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.860071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr" event={"ID":"48be81c0-979b-434b-a155-dd7f12bb973f","Type":"ContainerStarted","Data":"6cc1d362c437fe26c7a2fa54875b19448e2e91fa483963befb25597a8ab3ae3e"}
Jul 15 14:15:00.860310 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.860127    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr" event={"ID":"48be81c0-979b-434b-a155-dd7f12bb973f","Type":"ContainerStarted","Data":"57609115354bdb964f2c4ec14b90cf42a3f7e063c8c091b0376fba4dc4b5f14f"}
Jul 15 14:15:00.861387 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.861362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-rp82r" event={"ID":"e13dffdf-5156-4015-b86f-7c5c54d7e089","Type":"ContainerStarted","Data":"c521f8f49efb78956022f45de7ee5ba3ba494e581f72c2144d33fdee16b35539"}
Jul 15 14:15:00.881795 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:00.881741    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr" podStartSLOduration=0.881723586 podStartE2EDuration="881.723586ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:15:00.881147349 +0000 UTC m=+2745.224082454" watchObservedRunningTime="2026-07-15 14:15:00.881723586 +0000 UTC m=+2745.224658707"
Jul 15 14:15:01.869440 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:01.869352    2577 generic.go:356] "Generic (PLEG): container finished" podID="e13dffdf-5156-4015-b86f-7c5c54d7e089" containerID="a34e1b8903d947d2e9ced4fb876a607a6ee94d92e94fb36c7f01bfca4f16ee75" exitCode=0
Jul 15 14:15:01.869861 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:01.869442    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-rp82r" event={"ID":"e13dffdf-5156-4015-b86f-7c5c54d7e089","Type":"ContainerDied","Data":"a34e1b8903d947d2e9ced4fb876a607a6ee94d92e94fb36c7f01bfca4f16ee75"}
Jul 15 14:15:01.870916 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:01.870893    2577 generic.go:356] "Generic (PLEG): container finished" podID="39bd64e8-37c7-4ae0-89bb-c5e255752efc" containerID="ae32bc86ed3babc510cb39cd1763c6f13e068720b04e06a4afb7e2a48cb7d13b" exitCode=0
Jul 15 14:15:01.870986 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:01.870969    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78" event={"ID":"39bd64e8-37c7-4ae0-89bb-c5e255752efc","Type":"ContainerDied","Data":"ae32bc86ed3babc510cb39cd1763c6f13e068720b04e06a4afb7e2a48cb7d13b"}
Jul 15 14:15:01.872365 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:01.872338    2577 generic.go:356] "Generic (PLEG): container finished" podID="48be81c0-979b-434b-a155-dd7f12bb973f" containerID="6cc1d362c437fe26c7a2fa54875b19448e2e91fa483963befb25597a8ab3ae3e" exitCode=0
Jul 15 14:15:01.872472 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:01.872363    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr" event={"ID":"48be81c0-979b-434b-a155-dd7f12bb973f","Type":"ContainerDied","Data":"6cc1d362c437fe26c7a2fa54875b19448e2e91fa483963befb25597a8ab3ae3e"}
Jul 15 14:15:03.035667 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.035409    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78"
Jul 15 14:15:03.082301 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.082252    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr"
Jul 15 14:15:03.094119 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.094094    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-rp82r"
Jul 15 14:15:03.884584 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.884551    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-rp82r"
Jul 15 14:15:03.884776 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.884586    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735415-rp82r" event={"ID":"e13dffdf-5156-4015-b86f-7c5c54d7e089","Type":"ContainerDied","Data":"c521f8f49efb78956022f45de7ee5ba3ba494e581f72c2144d33fdee16b35539"}
Jul 15 14:15:03.884776 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.884637    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c521f8f49efb78956022f45de7ee5ba3ba494e581f72c2144d33fdee16b35539"
Jul 15 14:15:03.885902 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.885873    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78"
Jul 15 14:15:03.885902 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.885887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78" event={"ID":"39bd64e8-37c7-4ae0-89bb-c5e255752efc","Type":"ContainerDied","Data":"8300e9676f1cdbb56b2678610c57eb9fbba07c316aaa31d5dea483f2925c2a0b"}
Jul 15 14:15:03.886079 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.885923    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8300e9676f1cdbb56b2678610c57eb9fbba07c316aaa31d5dea483f2925c2a0b"
Jul 15 14:15:03.887240 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.887221    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr"
Jul 15 14:15:03.887240 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.887231    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr" event={"ID":"48be81c0-979b-434b-a155-dd7f12bb973f","Type":"ContainerDied","Data":"57609115354bdb964f2c4ec14b90cf42a3f7e063c8c091b0376fba4dc4b5f14f"}
Jul 15 14:15:03.887395 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:03.887297    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57609115354bdb964f2c4ec14b90cf42a3f7e063c8c091b0376fba4dc4b5f14f"
Jul 15 14:15:06.012704 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:06.012662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:15:07.001969 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:07.001938    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:15:11.488430 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.488395    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"]
Jul 15 14:15:11.488840 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.488825    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48be81c0-979b-434b-a155-dd7f12bb973f" containerName="cleanup"
Jul 15 14:15:11.488894 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.488844    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="48be81c0-979b-434b-a155-dd7f12bb973f" containerName="cleanup"
Jul 15 14:15:11.488894 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.488852    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39bd64e8-37c7-4ae0-89bb-c5e255752efc" containerName="cleanup"
Jul 15 14:15:11.488894 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.488857    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="39bd64e8-37c7-4ae0-89bb-c5e255752efc" containerName="cleanup"
Jul 15 14:15:11.488894 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.488866    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e13dffdf-5156-4015-b86f-7c5c54d7e089" containerName="cleanup"
Jul 15 14:15:11.488894 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.488873    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e13dffdf-5156-4015-b86f-7c5c54d7e089" containerName="cleanup"
Jul 15 14:15:11.489071 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.488934    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="39bd64e8-37c7-4ae0-89bb-c5e255752efc" containerName="cleanup"
Jul 15 14:15:11.489071 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.488941    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="48be81c0-979b-434b-a155-dd7f12bb973f" containerName="cleanup"
Jul 15 14:15:11.489071 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.488948    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e13dffdf-5156-4015-b86f-7c5c54d7e089" containerName="cleanup"
Jul 15 14:15:11.493205 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.493183    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.496000 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.495979    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2723e9\"/\"kube-root-ca.crt\""
Jul 15 14:15:11.497114 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.497089    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2723e9\"/\"openshift-service-ca.crt\""
Jul 15 14:15:11.497242 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.497142    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2723e9\"/\"default-dockercfg-c96r6\""
Jul 15 14:15:11.497242 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.497193    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2723e9\"/\"rate-test-model-2723e9-kserve-self-signed-certs\""
Jul 15 14:15:11.507123 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.507093    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"]
Jul 15 14:15:11.518332 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.518299    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:15:11.611217 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.611181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-tmp-dir\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.611429 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.611221    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-home\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.611429 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.611247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-dshm\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.611429 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.611327    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-model-cache\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.611429 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.611390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6185c0f1-a859-4183-becd-470981121555-tls-certs\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.611572 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.611443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbkms\" (UniqueName: \"kubernetes.io/projected/6185c0f1-a859-4183-becd-470981121555-kube-api-access-cbkms\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.712483 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.712439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-tmp-dir\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.712483 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.712484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-home\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.712726 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.712503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-dshm\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.712726 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.712532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-model-cache\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.712726 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.712577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6185c0f1-a859-4183-becd-470981121555-tls-certs\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.712726 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.712617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbkms\" (UniqueName: \"kubernetes.io/projected/6185c0f1-a859-4183-becd-470981121555-kube-api-access-cbkms\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.712938 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.712912    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-tmp-dir\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.712993 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.712938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-home\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.712993 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.712970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-model-cache\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.715049 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.715032    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-dshm\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.715201 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.715184    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6185c0f1-a859-4183-becd-470981121555-tls-certs\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.721684 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.721659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbkms\" (UniqueName: \"kubernetes.io/projected/6185c0f1-a859-4183-becd-470981121555-kube-api-access-cbkms\") pod \"rate-test-model-2723e9-kserve-799d9489ff-6jmd8\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") " pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.804165 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.804136    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:11.949048 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:11.949022    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"]
Jul 15 14:15:11.951383 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:15:11.951353    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6185c0f1_a859_4183_becd_470981121555.slice/crio-4748500486a56e62369fc8e08b5852fc5c80ce9172dc573e533ca593ea8183c0 WatchSource:0}: Error finding container 4748500486a56e62369fc8e08b5852fc5c80ce9172dc573e533ca593ea8183c0: Status 404 returned error can't find the container with id 4748500486a56e62369fc8e08b5852fc5c80ce9172dc573e533ca593ea8183c0
Jul 15 14:15:12.936741 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:12.936701    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8" event={"ID":"6185c0f1-a859-4183-becd-470981121555","Type":"ContainerStarted","Data":"e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed"}
Jul 15 14:15:12.936741 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:12.936741    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8" event={"ID":"6185c0f1-a859-4183-becd-470981121555","Type":"ContainerStarted","Data":"4748500486a56e62369fc8e08b5852fc5c80ce9172dc573e533ca593ea8183c0"}
Jul 15 14:15:12.955002 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:12.954938    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8" podStartSLOduration=1.9549165419999999 podStartE2EDuration="1.954916542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:15:12.954574874 +0000 UTC m=+2757.297509977" watchObservedRunningTime="2026-07-15 14:15:12.954916542 +0000 UTC m=+2757.297851622"
Jul 15 14:15:21.405563 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:21.405531    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:15:21.502783 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:21.502746    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:15:21.804812 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:21.804775    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:21.805007 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:21.804925    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:21.817446 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:21.817421    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:21.993837 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:21.993803    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:15:29.524128 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:29.524083    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:15:30.871600 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:30.871569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:15:59.232127 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:15:59.232059    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:16:50.006037 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:16:50.005997    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"]
Jul 15 14:16:50.020240 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:16:50.020195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:16:50.889498 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:16:50.889456    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:05.027429 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.027381    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8" podUID="6185c0f1-a859-4183-becd-470981121555" containerName="main" containerID="cri-o://e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed" gracePeriod=45
Jul 15 14:17:05.276891 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.276867    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:17:05.365204 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.365120    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-home\") pod \"6185c0f1-a859-4183-becd-470981121555\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") "
Jul 15 14:17:05.365204 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.365174    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-model-cache\") pod \"6185c0f1-a859-4183-becd-470981121555\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") "
Jul 15 14:17:05.365204 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.365200    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6185c0f1-a859-4183-becd-470981121555-tls-certs\") pod \"6185c0f1-a859-4183-becd-470981121555\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") "
Jul 15 14:17:05.365581 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.365229    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-tmp-dir\") pod \"6185c0f1-a859-4183-becd-470981121555\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") "
Jul 15 14:17:05.365581 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.365253    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-dshm\") pod \"6185c0f1-a859-4183-becd-470981121555\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") "
Jul 15 14:17:05.365581 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.365306    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbkms\" (UniqueName: \"kubernetes.io/projected/6185c0f1-a859-4183-becd-470981121555-kube-api-access-cbkms\") pod \"6185c0f1-a859-4183-becd-470981121555\" (UID: \"6185c0f1-a859-4183-becd-470981121555\") "
Jul 15 14:17:05.365581 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.365477    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-home" (OuterVolumeSpecName: "home") pod "6185c0f1-a859-4183-becd-470981121555" (UID: "6185c0f1-a859-4183-becd-470981121555"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:17:05.365581 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.365525    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-model-cache" (OuterVolumeSpecName: "model-cache") pod "6185c0f1-a859-4183-becd-470981121555" (UID: "6185c0f1-a859-4183-becd-470981121555"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:17:05.365581 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.365558    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-home\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:05.365817 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.365581    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6185c0f1-a859-4183-becd-470981121555" (UID: "6185c0f1-a859-4183-becd-470981121555"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:17:05.367779 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.367751    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-dshm" (OuterVolumeSpecName: "dshm") pod "6185c0f1-a859-4183-becd-470981121555" (UID: "6185c0f1-a859-4183-becd-470981121555"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 14:17:05.367779 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.367777    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6185c0f1-a859-4183-becd-470981121555-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6185c0f1-a859-4183-becd-470981121555" (UID: "6185c0f1-a859-4183-becd-470981121555"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:17:05.367922 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.367805    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6185c0f1-a859-4183-becd-470981121555-kube-api-access-cbkms" (OuterVolumeSpecName: "kube-api-access-cbkms") pod "6185c0f1-a859-4183-becd-470981121555" (UID: "6185c0f1-a859-4183-becd-470981121555"). InnerVolumeSpecName "kube-api-access-cbkms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:17:05.466663 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.466608    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-model-cache\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:05.466663 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.466657    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6185c0f1-a859-4183-becd-470981121555-tls-certs\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:05.466663 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.466669    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-tmp-dir\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:05.466663 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.466680    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6185c0f1-a859-4183-becd-470981121555-dshm\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:05.466948 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.466693    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbkms\" (UniqueName: \"kubernetes.io/projected/6185c0f1-a859-4183-becd-470981121555-kube-api-access-cbkms\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:05.517426 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.517393    2577 generic.go:356] "Generic (PLEG): container finished" podID="6185c0f1-a859-4183-becd-470981121555" containerID="e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed" exitCode=0
Jul 15 14:17:05.517620 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.517434    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"
Jul 15 14:17:05.517620 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.517485    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8" event={"ID":"6185c0f1-a859-4183-becd-470981121555","Type":"ContainerDied","Data":"e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed"}
Jul 15 14:17:05.517620 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.517537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8" event={"ID":"6185c0f1-a859-4183-becd-470981121555","Type":"ContainerDied","Data":"4748500486a56e62369fc8e08b5852fc5c80ce9172dc573e533ca593ea8183c0"}
Jul 15 14:17:05.517620 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.517562    2577 scope.go:117] "RemoveContainer" containerID="e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed"
Jul 15 14:17:05.526754 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.526734    2577 scope.go:117] "RemoveContainer" containerID="e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed"
Jul 15 14:17:05.527007 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:17:05.526985    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed\": container with ID starting with e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed not found: ID does not exist" containerID="e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed"
Jul 15 14:17:05.527071 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.527021    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed"} err="failed to get container status \"e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed\": rpc error: code = NotFound desc = could not find container \"e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed\": container with ID starting with e0bf8216988cb0ed621f773a92e5b039870066d8182531c4fc9f7aadc7e9e4ed not found: ID does not exist"
Jul 15 14:17:05.596838 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.596798    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"]
Jul 15 14:17:05.608606 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:05.608568    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-2723e9/rate-test-model-2723e9-kserve-799d9489ff-6jmd8"]
Jul 15 14:17:06.146560 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:06.146520    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6185c0f1-a859-4183-becd-470981121555" path="/var/lib/kubelet/pods/6185c0f1-a859-4183-becd-470981121555/volumes"
Jul 15 14:17:09.262249 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.262211    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6"]
Jul 15 14:17:09.262664 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.262599    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6185c0f1-a859-4183-becd-470981121555" containerName="main"
Jul 15 14:17:09.262664 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.262611    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6185c0f1-a859-4183-becd-470981121555" containerName="main"
Jul 15 14:17:09.262744 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.262676    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6185c0f1-a859-4183-becd-470981121555" containerName="main"
Jul 15 14:17:09.267323 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.267299    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6"
Jul 15 14:17:09.270565 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.270536    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 14:17:09.277163 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.277133    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6"]
Jul 15 14:17:09.401080 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.401041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e77dc9b2-7e46-41db-8058-7d82992509ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6\" (UID: \"e77dc9b2-7e46-41db-8058-7d82992509ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6"
Jul 15 14:17:09.502194 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.502160    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e77dc9b2-7e46-41db-8058-7d82992509ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6\" (UID: \"e77dc9b2-7e46-41db-8058-7d82992509ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6"
Jul 15 14:17:09.502858 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.502838    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e77dc9b2-7e46-41db-8058-7d82992509ea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6\" (UID: \"e77dc9b2-7e46-41db-8058-7d82992509ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6"
Jul 15 14:17:09.579102 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.579009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6"
Jul 15 14:17:09.718893 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.718866    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6"]
Jul 15 14:17:09.720944 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:17:09.720918    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode77dc9b2_7e46_41db_8058_7d82992509ea.slice/crio-0525536d46c053014fc38acf1ce9a6982216fb2e53d437933b3d405b4f38a9de WatchSource:0}: Error finding container 0525536d46c053014fc38acf1ce9a6982216fb2e53d437933b3d405b4f38a9de: Status 404 returned error can't find the container with id 0525536d46c053014fc38acf1ce9a6982216fb2e53d437933b3d405b4f38a9de
Jul 15 14:17:09.918440 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:09.918415    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:10.023407 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:10.023368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:10.546652 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:10.546615    2577 generic.go:356] "Generic (PLEG): container finished" podID="e77dc9b2-7e46-41db-8058-7d82992509ea" containerID="dd17ce8f8e888c2e674c8e25e8fa3e8e68912982f99963c422ae4675b27e8db2" exitCode=0
Jul 15 14:17:10.547065 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:10.546696    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6" event={"ID":"e77dc9b2-7e46-41db-8058-7d82992509ea","Type":"ContainerDied","Data":"dd17ce8f8e888c2e674c8e25e8fa3e8e68912982f99963c422ae4675b27e8db2"}
Jul 15 14:17:10.547065 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:10.546752    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6" event={"ID":"e77dc9b2-7e46-41db-8058-7d82992509ea","Type":"ContainerStarted","Data":"0525536d46c053014fc38acf1ce9a6982216fb2e53d437933b3d405b4f38a9de"}
Jul 15 14:17:11.688054 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:11.688030    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6"
Jul 15 14:17:11.824665 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:11.824566    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e77dc9b2-7e46-41db-8058-7d82992509ea-maas-api-service-ca\") pod \"e77dc9b2-7e46-41db-8058-7d82992509ea\" (UID: \"e77dc9b2-7e46-41db-8058-7d82992509ea\") "
Jul 15 14:17:11.824967 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:11.824942    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e77dc9b2-7e46-41db-8058-7d82992509ea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e77dc9b2-7e46-41db-8058-7d82992509ea" (UID: "e77dc9b2-7e46-41db-8058-7d82992509ea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:17:11.926118 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:11.926065    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e77dc9b2-7e46-41db-8058-7d82992509ea-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:12.558442 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:12.558406    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6" event={"ID":"e77dc9b2-7e46-41db-8058-7d82992509ea","Type":"ContainerDied","Data":"0525536d46c053014fc38acf1ce9a6982216fb2e53d437933b3d405b4f38a9de"}
Jul 15 14:17:12.558442 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:12.558440    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6"
Jul 15 14:17:12.558661 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:12.558487    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0525536d46c053014fc38acf1ce9a6982216fb2e53d437933b3d405b4f38a9de"
Jul 15 14:17:19.871668 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:19.871629    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"]
Jul 15 14:17:19.872126 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:19.871903    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm" podUID="c91fbced-410f-4545-ae70-633bcef9b5d9" containerName="maas-api" containerID="cri-o://06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497" gracePeriod=30
Jul 15 14:17:19.901049 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:19.901011    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr"]
Jul 15 14:17:19.904343 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:19.904317    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-2723e9-29735415-s86xr"]
Jul 15 14:17:19.921783 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:19.921752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:20.147406 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.147324    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48be81c0-979b-434b-a155-dd7f12bb973f" path="/var/lib/kubelet/pods/48be81c0-979b-434b-a155-dd7f12bb973f/volumes"
Jul 15 14:17:20.323083 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.323057    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:17:20.399343 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.399233    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c91fbced-410f-4545-ae70-633bcef9b5d9-maas-api-tls\") pod \"c91fbced-410f-4545-ae70-633bcef9b5d9\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") "
Jul 15 14:17:20.399343 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.399320    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p8qf\" (UniqueName: \"kubernetes.io/projected/c91fbced-410f-4545-ae70-633bcef9b5d9-kube-api-access-8p8qf\") pod \"c91fbced-410f-4545-ae70-633bcef9b5d9\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") "
Jul 15 14:17:20.399343 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.399342    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c91fbced-410f-4545-ae70-633bcef9b5d9-openshift-service-ca\") pod \"c91fbced-410f-4545-ae70-633bcef9b5d9\" (UID: \"c91fbced-410f-4545-ae70-633bcef9b5d9\") "
Jul 15 14:17:20.399770 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.399744    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c91fbced-410f-4545-ae70-633bcef9b5d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c91fbced-410f-4545-ae70-633bcef9b5d9" (UID: "c91fbced-410f-4545-ae70-633bcef9b5d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:17:20.401568 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.401542    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c91fbced-410f-4545-ae70-633bcef9b5d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c91fbced-410f-4545-ae70-633bcef9b5d9" (UID: "c91fbced-410f-4545-ae70-633bcef9b5d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:17:20.401674 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.401577    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c91fbced-410f-4545-ae70-633bcef9b5d9-kube-api-access-8p8qf" (OuterVolumeSpecName: "kube-api-access-8p8qf") pod "c91fbced-410f-4545-ae70-633bcef9b5d9" (UID: "c91fbced-410f-4545-ae70-633bcef9b5d9"). InnerVolumeSpecName "kube-api-access-8p8qf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:17:20.500631 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.500594    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c91fbced-410f-4545-ae70-633bcef9b5d9-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:20.500631 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.500627    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p8qf\" (UniqueName: \"kubernetes.io/projected/c91fbced-410f-4545-ae70-633bcef9b5d9-kube-api-access-8p8qf\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:20.500631 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.500638    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c91fbced-410f-4545-ae70-633bcef9b5d9-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:20.599506 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.599474    2577 generic.go:356] "Generic (PLEG): container finished" podID="c91fbced-410f-4545-ae70-633bcef9b5d9" containerID="06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497" exitCode=0
Jul 15 14:17:20.599707 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.599523    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"
Jul 15 14:17:20.599707 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.599525    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm" event={"ID":"c91fbced-410f-4545-ae70-633bcef9b5d9","Type":"ContainerDied","Data":"06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497"}
Jul 15 14:17:20.599707 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.599671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm" event={"ID":"c91fbced-410f-4545-ae70-633bcef9b5d9","Type":"ContainerDied","Data":"87ddfaac35a6639b7d98d90bb1db4915038634326e0151454d97189e1f98db7e"}
Jul 15 14:17:20.599884 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.599711    2577 scope.go:117] "RemoveContainer" containerID="06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497"
Jul 15 14:17:20.609464 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.609440    2577 scope.go:117] "RemoveContainer" containerID="06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497"
Jul 15 14:17:20.609730 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:17:20.609712    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497\": container with ID starting with 06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497 not found: ID does not exist" containerID="06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497"
Jul 15 14:17:20.609778 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.609738    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497"} err="failed to get container status \"06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497\": rpc error: code = NotFound desc = could not find container \"06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497\": container with ID starting with 06bac08ec4b6dcb8355aa614cef9e88d4fbed2bc2df45ad180872faca39fa497 not found: ID does not exist"
Jul 15 14:17:20.631192 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.631157    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"]
Jul 15 14:17:20.635649 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:20.635626    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-2723e9-5b877b48cc-npvcm"]
Jul 15 14:17:22.148492 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:22.148459    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c91fbced-410f-4545-ae70-633bcef9b5d9" path="/var/lib/kubelet/pods/c91fbced-410f-4545-ae70-633bcef9b5d9/volumes"
Jul 15 14:17:24.380195 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:24.380157    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:31.014911 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:31.014875    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:31.101447 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:31.101400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:35.139743 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.139703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:35.394514 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.394416    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:35.523905 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.523861    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm"]
Jul 15 14:17:35.524250 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.524236    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c91fbced-410f-4545-ae70-633bcef9b5d9" containerName="maas-api"
Jul 15 14:17:35.524326 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.524252    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c91fbced-410f-4545-ae70-633bcef9b5d9" containerName="maas-api"
Jul 15 14:17:35.524326 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.524284    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e77dc9b2-7e46-41db-8058-7d82992509ea" containerName="revoke-keys"
Jul 15 14:17:35.524326 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.524290    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e77dc9b2-7e46-41db-8058-7d82992509ea" containerName="revoke-keys"
Jul 15 14:17:35.524467 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.524349    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e77dc9b2-7e46-41db-8058-7d82992509ea" containerName="revoke-keys"
Jul 15 14:17:35.524467 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.524357    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c91fbced-410f-4545-ae70-633bcef9b5d9" containerName="maas-api"
Jul 15 14:17:35.527537 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.527515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm"
Jul 15 14:17:35.530151 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.530130    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v4sbs\""
Jul 15 14:17:35.536247 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.536222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm"]
Jul 15 14:17:35.626215 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.626173    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68eb2ac5-2ba3-4283-b76b-31db995bd789-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm\" (UID: \"68eb2ac5-2ba3-4283-b76b-31db995bd789\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm"
Jul 15 14:17:35.727290 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.727169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68eb2ac5-2ba3-4283-b76b-31db995bd789-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm\" (UID: \"68eb2ac5-2ba3-4283-b76b-31db995bd789\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm"
Jul 15 14:17:35.727873 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.727847    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68eb2ac5-2ba3-4283-b76b-31db995bd789-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm\" (UID: \"68eb2ac5-2ba3-4283-b76b-31db995bd789\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm"
Jul 15 14:17:35.839251 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.839210    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm"
Jul 15 14:17:35.983133 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:35.983103    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm"]
Jul 15 14:17:35.985739 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:17:35.985709    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68eb2ac5_2ba3_4283_b76b_31db995bd789.slice/crio-a7962bb070b2f9391c1b68f6f87683d898c382459883f51e418360ef10b3c60a WatchSource:0}: Error finding container a7962bb070b2f9391c1b68f6f87683d898c382459883f51e418360ef10b3c60a: Status 404 returned error can't find the container with id a7962bb070b2f9391c1b68f6f87683d898c382459883f51e418360ef10b3c60a
Jul 15 14:17:36.376733 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:36.376697    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:36.443962 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:36.443933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:36.683028 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:36.682937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm" event={"ID":"68eb2ac5-2ba3-4283-b76b-31db995bd789","Type":"ContainerStarted","Data":"e38a96c852f24a8178d250df9155c0f013a3563839d7aff537e9c9153b12dfcc"}
Jul 15 14:17:36.683028 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:36.682997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm" event={"ID":"68eb2ac5-2ba3-4283-b76b-31db995bd789","Type":"ContainerStarted","Data":"a7962bb070b2f9391c1b68f6f87683d898c382459883f51e418360ef10b3c60a"}
Jul 15 14:17:36.701516 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:36.701462    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm" podStartSLOduration=1.701445968 podStartE2EDuration="1.701445968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 14:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:17:36.699414279 +0000 UTC m=+2901.042349391" watchObservedRunningTime="2026-07-15 14:17:36.701445968 +0000 UTC m=+2901.044381048"
Jul 15 14:17:37.689757 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:37.689726    2577 generic.go:356] "Generic (PLEG): container finished" podID="68eb2ac5-2ba3-4283-b76b-31db995bd789" containerID="e38a96c852f24a8178d250df9155c0f013a3563839d7aff537e9c9153b12dfcc" exitCode=0
Jul 15 14:17:37.690199 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:37.689800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm" event={"ID":"68eb2ac5-2ba3-4283-b76b-31db995bd789","Type":"ContainerDied","Data":"e38a96c852f24a8178d250df9155c0f013a3563839d7aff537e9c9153b12dfcc"}
Jul 15 14:17:38.831279 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:38.831238    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm"
Jul 15 14:17:38.956560 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:38.956475    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68eb2ac5-2ba3-4283-b76b-31db995bd789-maas-api-service-ca\") pod \"68eb2ac5-2ba3-4283-b76b-31db995bd789\" (UID: \"68eb2ac5-2ba3-4283-b76b-31db995bd789\") "
Jul 15 14:17:38.956871 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:38.956850    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68eb2ac5-2ba3-4283-b76b-31db995bd789-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "68eb2ac5-2ba3-4283-b76b-31db995bd789" (UID: "68eb2ac5-2ba3-4283-b76b-31db995bd789"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:17:39.058014 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:39.057973    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68eb2ac5-2ba3-4283-b76b-31db995bd789-maas-api-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:39.702506 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:39.702454    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm"
Jul 15 14:17:39.702506 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:39.702463    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm" event={"ID":"68eb2ac5-2ba3-4283-b76b-31db995bd789","Type":"ContainerDied","Data":"a7962bb070b2f9391c1b68f6f87683d898c382459883f51e418360ef10b3c60a"}
Jul 15 14:17:39.702715 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:39.702544    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7962bb070b2f9391c1b68f6f87683d898c382459883f51e418360ef10b3c60a"
Jul 15 14:17:40.095592 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:40.095557    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:40.160491 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:40.160458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:46.228479 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.228430    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"]
Jul 15 14:17:46.229051 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.228692    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp" podUID="d7460b09-79ea-4ef0-b394-2dcf72f00ee8" containerName="maas-api" containerID="cri-o://eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b" gracePeriod=30
Jul 15 14:17:46.267083 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.267039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:46.281477 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.281449    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78"]
Jul 15 14:17:46.285422 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.285398    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfdd40-29735415-5jz78"]
Jul 15 14:17:46.482133 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.482068    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:17:46.521392 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.521365    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn7nk\" (UniqueName: \"kubernetes.io/projected/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-kube-api-access-bn7nk\") pod \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") "
Jul 15 14:17:46.521586 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.521475    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-openshift-service-ca\") pod \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") "
Jul 15 14:17:46.521586 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.521511    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-maas-api-tls\") pod \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\" (UID: \"d7460b09-79ea-4ef0-b394-2dcf72f00ee8\") "
Jul 15 14:17:46.521997 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.521959    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7460b09-79ea-4ef0-b394-2dcf72f00ee8" (UID: "d7460b09-79ea-4ef0-b394-2dcf72f00ee8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 14:17:46.524350 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.524309    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7460b09-79ea-4ef0-b394-2dcf72f00ee8" (UID: "d7460b09-79ea-4ef0-b394-2dcf72f00ee8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 14:17:46.524350 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.524324    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-kube-api-access-bn7nk" (OuterVolumeSpecName: "kube-api-access-bn7nk") pod "d7460b09-79ea-4ef0-b394-2dcf72f00ee8" (UID: "d7460b09-79ea-4ef0-b394-2dcf72f00ee8"). InnerVolumeSpecName "kube-api-access-bn7nk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 14:17:46.622523 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.622479    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-openshift-service-ca\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:46.622523 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.622517    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-maas-api-tls\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:46.622523 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.622526    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn7nk\" (UniqueName: \"kubernetes.io/projected/d7460b09-79ea-4ef0-b394-2dcf72f00ee8-kube-api-access-bn7nk\") on node \"ip-10-0-141-134.ec2.internal\" DevicePath \"\""
Jul 15 14:17:46.740059 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.739974    2577 generic.go:356] "Generic (PLEG): container finished" podID="d7460b09-79ea-4ef0-b394-2dcf72f00ee8" containerID="eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b" exitCode=0
Jul 15 14:17:46.740059 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.740020    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"
Jul 15 14:17:46.740233 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.740055    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp" event={"ID":"d7460b09-79ea-4ef0-b394-2dcf72f00ee8","Type":"ContainerDied","Data":"eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b"}
Jul 15 14:17:46.740233 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.740108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp" event={"ID":"d7460b09-79ea-4ef0-b394-2dcf72f00ee8","Type":"ContainerDied","Data":"40c96be1b3d32a0e2c352f82bd2ab491ea42a452a42dc09d285a09b11c3d1a6c"}
Jul 15 14:17:46.740233 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.740130    2577 scope.go:117] "RemoveContainer" containerID="eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b"
Jul 15 14:17:46.749084 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.749066    2577 scope.go:117] "RemoveContainer" containerID="eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b"
Jul 15 14:17:46.749373 ip-10-0-141-134 kubenswrapper[2577]: E0715 14:17:46.749348    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b\": container with ID starting with eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b not found: ID does not exist" containerID="eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b"
Jul 15 14:17:46.749470 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.749379    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b"} err="failed to get container status \"eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b\": rpc error: code = NotFound desc = could not find container \"eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b\": container with ID starting with eda964fa2353c3e46313c62bb61497450c95c5064974a66b02a56a88e0e8a84b not found: ID does not exist"
Jul 15 14:17:46.763473 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.763444    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"]
Jul 15 14:17:46.767329 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:46.767306    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfdd40-5c5bff6b56-ft4jp"]
Jul 15 14:17:48.146486 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:48.146448    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39bd64e8-37c7-4ae0-89bb-c5e255752efc" path="/var/lib/kubelet/pods/39bd64e8-37c7-4ae0-89bb-c5e255752efc/volumes"
Jul 15 14:17:48.146969 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:48.146944    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7460b09-79ea-4ef0-b394-2dcf72f00ee8" path="/var/lib/kubelet/pods/d7460b09-79ea-4ef0-b394-2dcf72f00ee8/volumes"
Jul 15 14:17:50.673390 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:50.673358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:57.264454 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:57.264418    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:17:57.355019 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:17:57.354983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:18:00.615226 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:00.615195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:18:01.375516 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:01.375485    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:18:01.609117 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:01.609089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:18:08.214049 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:08.214018    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-622x9_63558db3-414e-4b6f-b6a4-815cd0b91d44/manager/0.log"
Jul 15 14:18:08.482588 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:08.482516    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-577b8d5d55-bz7rm_de9faf9e-d48b-4834-8a68-292bb362d81f/manager/0.log"
Jul 15 14:18:08.611458 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:08.611433    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nstjz_4065938b-bc79-459e-b106-815157f1dedb/server/0.log"
Jul 15 14:18:08.734848 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:08.734774    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vrr7r_0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c/manager/0.log"
Jul 15 14:18:09.160872 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:09.160847    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84978f4d9d-qm7vt_08f760a9-1e2a-40fe-b054-a71fbddf46a5/copy-manifests/0.log"
Jul 15 14:18:09.168387 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:09.168366    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84978f4d9d-qm7vt_08f760a9-1e2a-40fe-b054-a71fbddf46a5/manager/0.log"
Jul 15 14:18:09.592287 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:09.592235    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-868cd55b6f-h6c4z_51e5cf3e-aeb7-4bd1-8406-1b721a84a206/maas-api/0.log"
Jul 15 14:18:09.722363 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:09.722333    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735385-xkmrq_f8c010e1-c41f-4a27-bd85-3f50bfaa1dc7/cleanup/0.log"
Jul 15 14:18:09.855196 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:09.855108    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735400-tb4mh_33c2521c-b6cb-4cbd-a842-99e61e7c963a/cleanup/0.log"
Jul 15 14:18:09.989477 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:09.989446    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735415-rp82r_e13dffdf-5156-4015-b86f-7c5c54d7e089/cleanup/0.log"
Jul 15 14:18:10.118108 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:10.118030    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-044bd9bf-242nw_fb0fe70c-0cf9-421b-84c0-c84998cb25cc/revoke-keys/0.log"
Jul 15 14:18:10.252064 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:10.252029    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-0e9155d7-8d9sf_ae91d8cb-ede2-48fa-90e8-9dc2e9b4277c/revoke-keys/0.log"
Jul 15 14:18:10.372251 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:10.372178    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-69a378-nzbjx_2fc3ae06-9408-44ca-b81c-81517290bbc5/revoke-keys/0.log"
Jul 15 14:18:10.506863 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:10.506831    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-f79741-xtwwn_01804600-9d29-4036-9a77-2083eb7bbbcc/revoke-keys/0.log"
Jul 15 14:18:10.634575 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:10.634504    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-6ff521e6-p5b7b_23a6eea2-f7c6-4fe1-b4b5-f9456a4508f4/revoke-keys/0.log"
Jul 15 14:18:10.763747 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:10.763718    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-16550d-6ghc4_38412d9f-0a05-4a56-a6e4-e6a1c094e018/revoke-keys/0.log"
Jul 15 14:18:10.885154 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:10.885077    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-59923e-6dpgm_e501a9fe-9768-4165-a24e-836decc5676d/revoke-keys/0.log"
Jul 15 14:18:11.023941 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:11.023914    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-65951df1-8pj8r_036f5ce0-df20-4e4f-b7aa-9afde4618b1b/revoke-keys/0.log"
Jul 15 14:18:11.145526 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:11.145456    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-b404f66f-qwtnj_156ef1ea-8905-4016-b13d-acba86b69628/revoke-keys/0.log"
Jul 15 14:18:11.261530 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:11.261502    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-c382ffa7-nkd98_d2e5d016-26de-496c-9c73-bea3d7a1dc70/revoke-keys/0.log"
Jul 15 14:18:11.383159 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:11.383132    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-2723e9-2h5m6_e77dc9b2-7e46-41db-8058-7d82992509ea/revoke-keys/0.log"
Jul 15 14:18:11.508513 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:11.508491    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-bfdd40-fprcm_68eb2ac5-2ba3-4283-b76b-31db995bd789/revoke-keys/0.log"
Jul 15 14:18:11.625837 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:11.625806    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-l4sdx_6b3d1e88-0d97-4a1f-a397-f692d06ac53e/postgres/0.log"
Jul 15 14:18:13.329229 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:13.329197    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-t4xdr_e63500dd-7460-4158-bb85-7a8c23803c95/manager/0.log"
Jul 15 14:18:13.981612 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:13.981581    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9xmdh_be13401e-dab7-4f3c-825a-4a793deca875/manager/0.log"
Jul 15 14:18:14.502708 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:14.502676    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p82bj_026e9cca-d755-4041-bf73-03b2b2fc852d/discovery/0.log"
Jul 15 14:18:14.769961 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:14.769931    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9ddcd897f-p6rlv_cd422153-9424-4cac-9124-c950387f80ea/kube-auth-proxy/0.log"
Jul 15 14:18:15.046230 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:15.046149    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-5r5gk_8ac19ece-4c41-4a36-b9f3-c19f1235888c/payload-pre-processing/0.log"
Jul 15 14:18:15.174175 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:15.174149    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-nclmh_4303d8a9-263a-47c5-ae3a-01fba4ac6e42/payload-processing/0.log"
Jul 15 14:18:15.870144 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:15.870106    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vsxqh_f996e924-7444-4b53-aa47-ed17a3b3a8e8/main/0.log"
Jul 15 14:18:16.282941 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:16.282916    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-mpzq8_7c7df76b-e600-4f2e-8d7b-48e70a8a7929/main/0.log"
Jul 15 14:18:22.794200 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:22.794164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:18:22.867149 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:22.867118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:18:25.801031 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:25.801003    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xzbd5_e918381b-1b9b-414a-80cb-a2aeac624753/global-pull-secret-syncer/0.log"
Jul 15 14:18:25.851617 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:25.851590    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log"
Jul 15 14:18:25.989072 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:25.989034    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-134.ec2.internal_a01437e717367c58578eaaf88d68b2a5/haproxy/0.log"
Jul 15 14:18:31.793889 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:31.793847    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-t4xdr_e63500dd-7460-4158-bb85-7a8c23803c95/manager/0.log"
Jul 15 14:18:32.246641 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:32.246554    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9xmdh_be13401e-dab7-4f3c-825a-4a793deca875/manager/0.log"
Jul 15 14:18:34.373607 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:34.373581    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2dg4k_a7fe5049-3140-4010-8905-b03a84d94d73/node-exporter/0.log"
Jul 15 14:18:34.426742 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:34.426715    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2dg4k_a7fe5049-3140-4010-8905-b03a84d94d73/kube-rbac-proxy/0.log"
Jul 15 14:18:34.485027 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:34.485001    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2dg4k_a7fe5049-3140-4010-8905-b03a84d94d73/init-textfile/0.log"
Jul 15 14:18:35.790226 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.790196    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"]
Jul 15 14:18:35.790630 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.790608    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68eb2ac5-2ba3-4283-b76b-31db995bd789" containerName="revoke-keys"
Jul 15 14:18:35.790630 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.790625    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="68eb2ac5-2ba3-4283-b76b-31db995bd789" containerName="revoke-keys"
Jul 15 14:18:35.790716 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.790644    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7460b09-79ea-4ef0-b394-2dcf72f00ee8" containerName="maas-api"
Jul 15 14:18:35.790716 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.790650    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7460b09-79ea-4ef0-b394-2dcf72f00ee8" containerName="maas-api"
Jul 15 14:18:35.790716 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.790711    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7460b09-79ea-4ef0-b394-2dcf72f00ee8" containerName="maas-api"
Jul 15 14:18:35.790819 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.790722    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="68eb2ac5-2ba3-4283-b76b-31db995bd789" containerName="revoke-keys"
Jul 15 14:18:35.794059 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.794043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:35.796861 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.796841    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-858vm\"/\"openshift-service-ca.crt\""
Jul 15 14:18:35.797754 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.797735    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-858vm\"/\"default-dockercfg-cx228\""
Jul 15 14:18:35.797876 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.797780    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-858vm\"/\"kube-root-ca.crt\""
Jul 15 14:18:35.802410 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.802391    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"]
Jul 15 14:18:35.924444 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.924405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-sys\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:35.924444 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.924444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-podres\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:35.924657 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.924551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-lib-modules\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:35.924657 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.924579    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bsbs\" (UniqueName: \"kubernetes.io/projected/2f011e50-4a43-491d-844a-d895a2fbf039-kube-api-access-8bsbs\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:35.924657 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:35.924610    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-proc\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.025414 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.025379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-sys\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.025414 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.025416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-podres\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.025654 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.025453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-lib-modules\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.025654 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.025519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-sys\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.025654 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.025564    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-lib-modules\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.025654 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.025570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-podres\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.025654 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.025577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bsbs\" (UniqueName: \"kubernetes.io/projected/2f011e50-4a43-491d-844a-d895a2fbf039-kube-api-access-8bsbs\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.025654 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.025643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-proc\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.025845 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.025727    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2f011e50-4a43-491d-844a-d895a2fbf039-proc\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.034574 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.034552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bsbs\" (UniqueName: \"kubernetes.io/projected/2f011e50-4a43-491d-844a-d895a2fbf039-kube-api-access-8bsbs\") pod \"perf-node-gather-daemonset-6fpq2\" (UID: \"2f011e50-4a43-491d-844a-d895a2fbf039\") " pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.105415 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.105330    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:36.248779 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:36.248753    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"]
Jul 15 14:18:36.250712 ip-10-0-141-134 kubenswrapper[2577]: W0715 14:18:36.250678    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2f011e50_4a43_491d_844a_d895a2fbf039.slice/crio-a9880db91208e04e33eb97ae63fde36fe2f6e4539c63ed495ce947b98cb820ab WatchSource:0}: Error finding container a9880db91208e04e33eb97ae63fde36fe2f6e4539c63ed495ce947b98cb820ab: Status 404 returned error can't find the container with id a9880db91208e04e33eb97ae63fde36fe2f6e4539c63ed495ce947b98cb820ab
Jul 15 14:18:37.003152 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:37.003115    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2" event={"ID":"2f011e50-4a43-491d-844a-d895a2fbf039","Type":"ContainerStarted","Data":"631a6b51e38c616f1df83266c353ebca7cac187eb973adc56c6ff4f4e28224cb"}
Jul 15 14:18:37.003687 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:37.003175    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2" event={"ID":"2f011e50-4a43-491d-844a-d895a2fbf039","Type":"ContainerStarted","Data":"a9880db91208e04e33eb97ae63fde36fe2f6e4539c63ed495ce947b98cb820ab"}
Jul 15 14:18:37.003687 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:37.003241    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:37.024597 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:37.024548    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2" podStartSLOduration=2.024531228 podStartE2EDuration="2.024531228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 14:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 14:18:37.023045334 +0000 UTC m=+2961.365980415" watchObservedRunningTime="2026-07-15 14:18:37.024531228 +0000 UTC m=+2961.367466306"
Jul 15 14:18:37.664461 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:37.664430    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-4hhbg_6222ec40-0d4d-460c-9528-66af0c47c175/download-server/0.log"
Jul 15 14:18:38.249839 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:38.249806    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:18:38.351589 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:38.351557    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:18:39.209127 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:39.209094    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6mkwn_160c3910-c9da-44fd-a1c9-22fb19468b3d/konnectivity-agent/0.log" limit=1024
Jul 15 14:18:39.550592 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:39.550563    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j8lb8_19557185-bb6a-4cb2-97c0-71457f1f9eb8/dns/0.log"
Jul 15 14:18:39.573344 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:39.573317    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j8lb8_19557185-bb6a-4cb2-97c0-71457f1f9eb8/kube-rbac-proxy/0.log"
Jul 15 14:18:39.680423 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:39.680397    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gdtmh_cedb62b6-fd1f-4ccc-9b9e-987f001a3879/dns-node-resolver/0.log"
Jul 15 14:18:40.283829 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:40.283804    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5dd7fddbc6-bd6xz_522a352f-f06c-4132-968a-4c275d0cc547/registry/0.log"
Jul 15 14:18:40.374520 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:40.374494    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w7dvt_c79e9a2e-64e5-487c-b25b-07b6853db2cf/node-ca/0.log"
Jul 15 14:18:41.602496 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:41.602458    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p82bj_026e9cca-d755-4041-bf73-03b2b2fc852d/discovery/0.log"
Jul 15 14:18:41.660772 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:41.660749    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9ddcd897f-p6rlv_cd422153-9424-4cac-9124-c950387f80ea/kube-auth-proxy/0.log"
Jul 15 14:18:41.803585 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:41.803554    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-5r5gk_8ac19ece-4c41-4a36-b9f3-c19f1235888c/payload-pre-processing/0.log"
Jul 15 14:18:41.856920 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:41.856843    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-nclmh_4303d8a9-263a-47c5-ae3a-01fba4ac6e42/payload-processing/0.log"
Jul 15 14:18:42.700155 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:42.700126    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hg9g7_f4cbbdc9-d133-4aca-a034-b49b8d8fe789/serve-healthcheck-canary/0.log"
Jul 15 14:18:43.019989 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:43.019963    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-858vm/perf-node-gather-daemonset-6fpq2"
Jul 15 14:18:43.284661 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:43.284590    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rdb96_35e809cf-4d2d-4499-829f-c9e7ef493957/kube-rbac-proxy/0.log"
Jul 15 14:18:43.323582 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:43.323559    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rdb96_35e809cf-4d2d-4499-829f-c9e7ef493957/exporter/0.log"
Jul 15 14:18:43.350241 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:43.350217    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rdb96_35e809cf-4d2d-4499-829f-c9e7ef493957/extractor/0.log"
Jul 15 14:18:46.080766 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:46.080727    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-622x9_63558db3-414e-4b6f-b6a4-815cd0b91d44/manager/0.log"
Jul 15 14:18:46.333313 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:46.333212    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-577b8d5d55-bz7rm_de9faf9e-d48b-4834-8a68-292bb362d81f/manager/0.log"
Jul 15 14:18:46.356055 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:46.356031    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nstjz_4065938b-bc79-459e-b106-815157f1dedb/server/0.log"
Jul 15 14:18:46.433966 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:46.433936    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vrr7r_0fa97da8-0c9a-4f9e-afdc-8fb4ca210d4c/manager/0.log"
Jul 15 14:18:46.574187 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:46.574153    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p82bj_026e9cca-d755-4041-bf73-03b2b2fc852d/discovery/0.log"
Jul 15 14:18:46.610559 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:46.610490    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9ddcd897f-p6rlv_cd422153-9424-4cac-9124-c950387f80ea/kube-auth-proxy/0.log"
Jul 15 14:18:46.657594 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:46.657567    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84978f4d9d-qm7vt_08f760a9-1e2a-40fe-b054-a71fbddf46a5/manager/0.log"
Jul 15 14:18:46.691346 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:46.691321    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84978f4d9d-qm7vt_08f760a9-1e2a-40fe-b054-a71fbddf46a5/copy-manifests/0.log"
Jul 15 14:18:46.691749 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:46.691730    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-5r5gk_8ac19ece-4c41-4a36-b9f3-c19f1235888c/payload-pre-processing/0.log"
Jul 15 14:18:46.713004 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:46.712973    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-nclmh_4303d8a9-263a-47c5-ae3a-01fba4ac6e42/payload-processing/0.log"
Jul 15 14:18:48.299749 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:48.299714    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p82bj_026e9cca-d755-4041-bf73-03b2b2fc852d/discovery/0.log"
Jul 15 14:18:48.337848 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:48.337819    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9ddcd897f-p6rlv_cd422153-9424-4cac-9124-c950387f80ea/kube-auth-proxy/0.log"
Jul 15 14:18:48.422607 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:48.422581    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-5r5gk_8ac19ece-4c41-4a36-b9f3-c19f1235888c/payload-pre-processing/0.log"
Jul 15 14:18:48.445548 ip-10-0-141-134 kubenswrapper[2577]: I0715 14:18:48.445521    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-nclmh_4303d8a9-263a-47c5-ae3a-01fba4ac6e42/payload-processing/0.log"