Jul 30 14:48:14.230683 ip-10-0-133-92 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 14:48:14.542319 ip-10-0-133-92 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 14:48:14.542319 ip-10-0-133-92 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 14:48:14.542319 ip-10-0-133-92 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 14:48:14.542319 ip-10-0-133-92 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 14:48:14.542319 ip-10-0-133-92 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 14:48:14.544517 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.544350    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 14:48:14.551386 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551350    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:14.551386 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551381    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:14.551386 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551386    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:14.551386 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551389    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:14.551386 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551409    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551413    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551416    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551420    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551423    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551426    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551428    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551431    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551434    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551436    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551438    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551441    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551444    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551446    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551452    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551455    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551459    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551464    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551467    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551469    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:14.551610 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551472    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551475    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551477    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551480    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551483    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551485    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551488    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551490    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551493    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551495    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551497    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551500    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551502    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551505    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551508    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551511    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551514    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551516    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551519    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551522    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551524    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:14.552098 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551528    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551535    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551538    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551541    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551543    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551547    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551550    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551553    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551555    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551558    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551560    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551562    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551565    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551567    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551570    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551573    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551575    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551578    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551580    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551583    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:14.552640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551585    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551587    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551590    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551592    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551595    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551598    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551600    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551603    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551606    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551609    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551611    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551614    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551617    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551620    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551622    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551625    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551628    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551632    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551636    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:14.553114 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551639    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551642    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551646    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551650    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551653    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551655    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551658    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551661    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551664    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551666    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551669    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551671    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551673    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551676    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:14.553611 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.551678    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553699    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553708    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553711    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553714    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553717    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553722    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553725    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553729    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553733    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553736    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553739    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553742    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553744    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553747    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553749    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553752    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553754    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553758    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:14.553951 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553760    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553763    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553765    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553768    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553770    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553773    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553775    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553777    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553780    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553782    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553784    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553786    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553789    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553791    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553793    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553798    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553807    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553810    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553813    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553815    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:14.554443 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553818    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553820    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553824    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553827    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553829    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553832    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553834    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553836    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553839    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553841    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553844    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553846    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553852    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553854    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553857    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553859    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553862    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553864    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553867    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553869    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:14.554938 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553872    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553875    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553877    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553879    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553882    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553884    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553886    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553889    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553892    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553894    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553897    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553899    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553902    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553904    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553906    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553909    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553911    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553914    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553916    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553918    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:14.555482 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553921    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553923    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553926    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553930    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553933    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553938    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553942    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553945    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553947    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553950    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553952    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553955    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553958    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553960    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553963    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553966    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553968    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553971    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553973    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553976    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:14.555963 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.553978    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554364    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554368    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554371    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554373    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554377    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554381    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554384    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554387    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554389    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554412    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554415    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554418    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554420    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554423    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554426    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554430    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554434    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554437    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:14.556568 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554439    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554442    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554444    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554447    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554450    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554452    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554455    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554458    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554461    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554463    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554465    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554468    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554471    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554474    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554476    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554479    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554481    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554483    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554486    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554488    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:14.557079 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554491    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554493    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554496    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554498    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554501    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554503    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554505    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554508    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554512    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554515    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554518    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554521    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554523    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554526    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554528    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554531    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554533    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554536    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554539    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:14.557583 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554541    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554544    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554546    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554548    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554551    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554554    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554556    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554558    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554561    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554563    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554566    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554569    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554571    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554574    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554576    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554578    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554581    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554583    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554586    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554588    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:14.558045 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554590    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554593    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554595    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554598    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554600    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554603    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554605    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554610    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554612    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554614    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554617    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554619    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554622    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554625    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554627    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554630    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554633    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554635    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554637    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554640    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:14.558574 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554642    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.554644    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554719    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554726    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554731    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554736    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554740    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554743    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554759    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554764    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554767    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554770    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554774    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554777    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554780    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554783    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554786    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554789    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554792    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554795    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554800    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554802    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554805    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554808    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 14:48:14.559067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554812    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554815    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554818    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554821    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554824    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554827    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554829    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554832    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554835    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554838    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554843    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554845    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554848    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554851    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554854    2582 flags.go:64] FLAG: --enable-server="true"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554856    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554861    2582 flags.go:64] FLAG: --event-burst="100"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554864    2582 flags.go:64] FLAG: --event-qps="50"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554866    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554869    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554872    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554876    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554878    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554881    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554884    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 30 14:48:14.559688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554886    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554889    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554892    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554895    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554898    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554900    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554903    2582 flags.go:64] FLAG: --feature-gates=""
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554907    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554910    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554912    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554916    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554918    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554922    2582 flags.go:64] FLAG: --help="false"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554925    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554927    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554930    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554933    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554936    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554939    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554942    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554945    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554947    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554950    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 14:48:14.560277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554953    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554956    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554959    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554961    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554964    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554966    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554969    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554971    2582 flags.go:64] FLAG: --lock-file=""
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554974    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554977    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554980    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554986    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554989    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554991    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554994    2582 flags.go:64] FLAG: --logging-format="text"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.554997    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555000    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555003    2582 flags.go:64] FLAG: --manifest-url=""
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555006    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555010    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555015    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555019    2582 flags.go:64] FLAG: --max-pods="110"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555022    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555025    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555028    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 14:48:14.560877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555031    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555034    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555036    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555039    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555047    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555050    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555053    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555056    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555059    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555064    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555067    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555070    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555073    2582 flags.go:64] FLAG: --port="10250"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555076    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555078    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00415f82b3854cd8d"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555081    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555084    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555087    2582 flags.go:64] FLAG: --register-node="true"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555090    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555094    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555097    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555100    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555103    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555107    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555110    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 14:48:14.561491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555113    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555116    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555120    2582 flags.go:64] FLAG: --runonce="false"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555122    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555126    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555129    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555131    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555134    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555137    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555140    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555143    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555146    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555149    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555152    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555154    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555157    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555160    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555162    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555168    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555170    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555173    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555177    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555180    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555183    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555185    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 14:48:14.562201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555188    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555191    2582 flags.go:64] FLAG: --v="2"
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555194    2582 flags.go:64] FLAG: --version="false"
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555199    2582 flags.go:64] FLAG: --vmodule=""
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555202    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.555207    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555322    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555326    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555329    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555333    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555336    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555338    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555341    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555344    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555346    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555349    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555352    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555354    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555357    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555359    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555361    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:14.562823 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555374    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555379    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555382    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555385    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555387    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555406    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555410    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555412    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555415    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555418    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555420    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555422    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555425    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555428    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555430    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555434    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555438    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555440    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555443    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:14.563347 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555445    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555450    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555452    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555455    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555457    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555460    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555462    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555465    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555467    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555470    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555474    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555476    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555479    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555481    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555484    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555486    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555489    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555492    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555494    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555497    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:14.563837 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555499    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555502    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555506    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555509    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555512    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555515    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555517    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555520    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555523    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555527    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555529    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555532    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555534    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555538    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555541    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555543    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555545    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555548    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555551    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:14.564324 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555553    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555556    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555558    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555560    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555563    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555566    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555568    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555570    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555573    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555575    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555577    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555580    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555582    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555584    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555587    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555591    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555594    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555596    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555599    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555602    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:14.564799 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555605    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555608    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555613    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555616    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555618    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.555621    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.556290    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.563121    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.563140    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.563784    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.563792    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563851    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563857    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563861    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:14.565288 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563866    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563907    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563911    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563914    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563916    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563919    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563922    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563925    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563928    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563932    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563934    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563937    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563940    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563942    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563945    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563947    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563950    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563953    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563956    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563958    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:14.565729 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563961    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563965    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563967    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563970    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563972    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563975    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563978    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563980    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563983    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563985    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563988    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563990    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563992    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563995    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.563998    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564000    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564003    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564005    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564008    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564010    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:14.566217 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564013    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564015    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564018    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564022    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564025    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564028    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564031    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564034    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564036    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564039    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564041    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564044    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564046    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564049    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564051    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564054    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564056    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564059    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564063    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:14.566712 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564066    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564068    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564072    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564074    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564077    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564079    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564082    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564084    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564087    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564089    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564092    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564094    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564097    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564099    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564102    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564104    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564106    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564109    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564111    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564113    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:14.567225 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564116    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564118    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564121    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564123    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564126    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564128    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564130    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564133    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564137    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564141    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564143    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564146    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564149    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564152    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564156    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564159    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:14.567741 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564162    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.564167    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564284    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564289    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564292    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564294    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564297    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564300    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564303    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564305    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564308    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564310    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564313    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564315    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564318    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 14:48:14.568152 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564320    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564323    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564325    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564328    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564331    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564333    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564336    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564338    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564341    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564343    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564347    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564349    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564352    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564355    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564357    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564360    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564362    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564365    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564367    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564370    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 14:48:14.568543 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564372    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564375    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564378    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564381    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564384    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564387    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564409    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564414    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564418    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564422    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564428    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564432    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564435    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564438    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564441    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564443    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564446    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564448    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564451    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564453    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 14:48:14.569024 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564456    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564458    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564461    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564465    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564468    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564470    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564472    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564475    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564477    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564480    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564482    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564484    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564487    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564489    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564492    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564494    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564496    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564499    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564501    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 14:48:14.569646 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564504    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564506    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564508    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564511    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564513    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564515    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564518    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564520    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564522    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564525    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564527    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564529    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564532    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564534    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564537    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564539    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564542    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564545    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564547    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564550    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 14:48:14.570101 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564552    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564554    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564557    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564559    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564562    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564565    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:14.564568    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.564573    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.564696    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.566791    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.567989    2582 server.go:1019] "Starting client certificate rotation"
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.568098    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.568129    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 14:48:14.570622 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.570431    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 14:48:14.575208 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.575180    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 14:48:14.586801 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.586779    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 30 14:48:14.588943 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.588928    2582 log.go:25] "Validated CRI v1 image API"
Jul 30 14:48:14.589741 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.589724    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 14:48:14.591967 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.591943    2582 fs.go:135] Filesystem UUIDs: map[23848d0b-7140-4342-8f56-637ba53fec42:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7f4a36b2-6dca-4c27-a0db-6ba17e7c3b39:/dev/nvme0n1p4]
Jul 30 14:48:14.592049 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.591965    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 14:48:14.596492 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.596466    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 14:48:14.596874 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.596769    2582 manager.go:216] Machine: {Timestamp:2026-07-30 14:48:14.596087336 +0000 UTC m=+0.269533440 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100132 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28e9013f572a98dd504b2d129ca8b8 SystemUUID:ec28e901-3f57-2a98-dd50-4b2d129ca8b8 BootID:0db8807d-2a6b-44bb-91ae-90e17566d89a Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a1:72:40:06:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a1:72:40:06:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:c1:21:a5:ce:ed Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 14:48:14.596874 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.596870    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 14:48:14.596979 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.596950    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 14:48:14.599090 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.599060    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 14:48:14.599232 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.599094    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-92.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 14:48:14.600155 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.600144    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 14:48:14.600195 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.600157    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 14:48:14.600195 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.600165    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 14:48:14.600910 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.600899    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 14:48:14.601664 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.601654    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 14:48:14.604711 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.604698    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 14:48:14.604864 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.604853    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 14:48:14.604898 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.604884    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 14:48:14.604898 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.604895    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 14:48:14.604948 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.604908    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 14:48:14.604948 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.604923    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 30 14:48:14.604948 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.604936    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 14:48:14.605480 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.605460    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 14:48:14.605572 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.605497    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 14:48:14.607899 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.607880    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 14:48:14.609679 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.609666    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 14:48:14.609736 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.609696    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 14:48:14.611130 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611113    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 14:48:14.611130 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611130    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 14:48:14.611196 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611136    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 14:48:14.611196 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611142    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 14:48:14.611196 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611151    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 14:48:14.611196 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611156    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 14:48:14.611196 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611164    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 14:48:14.611196 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611170    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 14:48:14.611196 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611182    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 14:48:14.611196 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611188    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 14:48:14.611439 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611203    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 14:48:14.611439 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.611208    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 14:48:14.612087 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.612069    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 14:48:14.612087 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.612084    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 14:48:14.614025 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.614004    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-92.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 14:48:14.614497 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.614471    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 14:48:14.614497 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.614471    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 14:48:14.615494 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.615477    2582 server.go:1268] "Started kubelet"
Jul 30 14:48:14.615619 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.615563    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 14:48:14.615681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.615601    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 14:48:14.615729 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.615655    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 14:48:14.616408 ip-10-0-133-92 systemd[1]: Started Kubernetes Kubelet.
Jul 30 14:48:14.617549 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.617530    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 14:48:14.619753 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.619734    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 14:48:14.621770 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.620969    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-92.ec2.internal.18c718a9bf675275  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-92.ec2.internal,UID:ip-10-0-133-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-92.ec2.internal,},FirstTimestamp:2026-07-30 14:48:14.615450229 +0000 UTC m=+0.288896333,LastTimestamp:2026-07-30 14:48:14.615450229 +0000 UTC m=+0.288896333,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-92.ec2.internal,}"
Jul 30 14:48:14.622958 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.622941    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dv2xr"
Jul 30 14:48:14.623047 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.623010    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 14:48:14.630975 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.630948    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dv2xr"
Jul 30 14:48:14.635455 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.635435    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 14:48:14.635651 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.635536    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 14:48:14.635806 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.635787    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:14.636525 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636504    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 14:48:14.636525 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636526    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 14:48:14.636670 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636509    2582 factory.go:55] Registering systemd factory
Jul 30 14:48:14.636670 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636549    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 30 14:48:14.636670 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636504    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 14:48:14.636670 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636609    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 14:48:14.636670 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636619    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 14:48:14.637055 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636819    2582 factory.go:153] Registering CRI-O factory
Jul 30 14:48:14.637055 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636829    2582 factory.go:223] Registration of the crio container factory successfully
Jul 30 14:48:14.637055 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636874    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 14:48:14.637055 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636892    2582 factory.go:103] Registering Raw factory
Jul 30 14:48:14.637055 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.636904    2582 manager.go:1196] Started watching for new ooms in manager
Jul 30 14:48:14.637463 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.637445    2582 manager.go:349] Starting recovery of all containers
Jul 30 14:48:14.646687 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.646667    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:14.647615 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.647447    2582 manager.go:354] Recovery completed
Jul 30 14:48:14.649784 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.649764    2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-92.ec2.internal\" not found" node="ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.652262 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.652248    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:14.654202 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.654188    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:14.654275 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.654219    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:14.654275 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.654232    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:14.654848 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.654835    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 14:48:14.654848 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.654846    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 14:48:14.654921 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.654863    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 14:48:14.654921 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.654886    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:14.655681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.655667    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:14.655736 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.655691    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:14.655736 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.655702    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:14.657598 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.657585    2582 policy_none.go:49] "None policy: Start"
Jul 30 14:48:14.657632 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.657603    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 14:48:14.657632 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.657630    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 14:48:14.659523 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.659513    2582 policy_none.go:47] "Start"
Jul 30 14:48:14.700095 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.700073    2582 manager.go:337] "Starting Device Plugin manager"
Jul 30 14:48:14.700223 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.700113    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 14:48:14.700223 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.700124    2582 server.go:87] "Starting device plugin registration server"
Jul 30 14:48:14.700467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.700440    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 14:48:14.700531 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.700468    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 14:48:14.700579 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.700538    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 14:48:14.700643 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.700630    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 14:48:14.700685 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.700644    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 14:48:14.701174 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.701152    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 14:48:14.701250 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.701216    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:14.716432 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.716409    2582 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2618 into the system container \"/system.slice\": <nil>"
Jul 30 14:48:14.799823 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.799721    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 14:48:14.802007 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.800956    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:14.802007 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.801068    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 14:48:14.802007 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.801105    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 14:48:14.802007 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.801127    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 14:48:14.802007 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.801165    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 14:48:14.802007 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.801944    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:14.802007 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.801970    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:14.802007 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.801986    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:14.802454 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.802021    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.803791 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.803772    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:14.822827 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.822804    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.822827 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.822830    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-92.ec2.internal\": node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:14.863705 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.863672    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:14.901784 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.901751    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal"]
Jul 30 14:48:14.901858 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.901837    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:14.902970 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.902951    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:14.903079 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.902981    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:14.903079 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.902991    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:14.905374 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.905361    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:14.905520 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.905505    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.905559 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.905539    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:14.906184 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.906167    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:14.906252 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.906195    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:14.906252 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.906207    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:14.906252 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.906235    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:14.906342 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.906262    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:14.906342 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.906276    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:14.908526 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.908509    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.908613 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.908540    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 14:48:14.909210 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.909195    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 14:48:14.909299 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.909222    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 14:48:14.909299 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.909237    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeHasSufficientPID"
Jul 30 14:48:14.923968 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.923948    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-92.ec2.internal\" not found" node="ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.934281 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.934265    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-92.ec2.internal\" not found" node="ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.938491 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.938474    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb369d9a4970133f5e28748fa24db29c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal\" (UID: \"cb369d9a4970133f5e28748fa24db29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.938564 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.938505    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb369d9a4970133f5e28748fa24db29c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal\" (UID: \"cb369d9a4970133f5e28748fa24db29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.938564 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:14.938541    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/958f474d6621921ad2d3c1a4ebe5848f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-92.ec2.internal\" (UID: \"958f474d6621921ad2d3c1a4ebe5848f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:14.963770 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:14.963740    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:15.038809 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.038779    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb369d9a4970133f5e28748fa24db29c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal\" (UID: \"cb369d9a4970133f5e28748fa24db29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:15.038898 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.038812    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/958f474d6621921ad2d3c1a4ebe5848f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-92.ec2.internal\" (UID: \"958f474d6621921ad2d3c1a4ebe5848f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:15.038898 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.038832    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb369d9a4970133f5e28748fa24db29c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal\" (UID: \"cb369d9a4970133f5e28748fa24db29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:15.038898 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.038879    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb369d9a4970133f5e28748fa24db29c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal\" (UID: \"cb369d9a4970133f5e28748fa24db29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:15.038898 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.038886    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb369d9a4970133f5e28748fa24db29c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal\" (UID: \"cb369d9a4970133f5e28748fa24db29c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:15.039018 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.038950    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/958f474d6621921ad2d3c1a4ebe5848f-config\") pod \"kube-apiserver-proxy-ip-10-0-133-92.ec2.internal\" (UID: \"958f474d6621921ad2d3c1a4ebe5848f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:15.063884 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:15.063819    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:15.164706 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:15.164660    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:15.226877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.226849    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:15.237938 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.237917    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:15.265153 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:15.265127    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:15.365802 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:15.365700    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:15.466211 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:15.466172    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:15.566756 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:15.566724    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:15.567812 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.567798    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 14:48:15.567958 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.567940    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 14:48:15.567998 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.567969    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 14:48:15.636202 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.636121    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 14:43:14 +0000 UTC" deadline="2028-03-14 18:01:04.490306583 +0000 UTC"
Jul 30 14:48:15.636202 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.636154    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14235h12m48.854158119s"
Jul 30 14:48:15.667557 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:15.667523    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:15.768163 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:15.768132    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-92.ec2.internal\" not found"
Jul 30 14:48:15.776640 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:15.776593    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb369d9a4970133f5e28748fa24db29c.slice/crio-2fd4cf593f57f0f0898ac33826513186399c0c70e71b75cfcc0faf27f4aaf1d5 WatchSource:0}: Error finding container 2fd4cf593f57f0f0898ac33826513186399c0c70e71b75cfcc0faf27f4aaf1d5: Status 404 returned error can't find the container with id 2fd4cf593f57f0f0898ac33826513186399c0c70e71b75cfcc0faf27f4aaf1d5
Jul 30 14:48:15.777059 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:15.777040    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958f474d6621921ad2d3c1a4ebe5848f.slice/crio-d050d17ccd50bf70450e338ea0701c51b734ea6d57a92d2e33fa281922301fa4 WatchSource:0}: Error finding container d050d17ccd50bf70450e338ea0701c51b734ea6d57a92d2e33fa281922301fa4: Status 404 returned error can't find the container with id d050d17ccd50bf70450e338ea0701c51b734ea6d57a92d2e33fa281922301fa4
Jul 30 14:48:15.780719 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.780702    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 14:48:15.799817 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.799793    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:15.804267 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.804227    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal" event={"ID":"cb369d9a4970133f5e28748fa24db29c","Type":"ContainerStarted","Data":"2fd4cf593f57f0f0898ac33826513186399c0c70e71b75cfcc0faf27f4aaf1d5"}
Jul 30 14:48:15.805179 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.805152    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal" event={"ID":"958f474d6621921ad2d3c1a4ebe5848f","Type":"ContainerStarted","Data":"d050d17ccd50bf70450e338ea0701c51b734ea6d57a92d2e33fa281922301fa4"}
Jul 30 14:48:15.836347 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.836317    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:15.846368 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.846347    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 14:48:15.847829 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.847815    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal"
Jul 30 14:48:15.853504 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.853490    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 14:48:15.879801 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:15.879773    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:16.381581 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.381552    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:16.506638 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.506598    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 14:48:16.606509 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.606476    2582 apiserver.go:52] "Watching apiserver"
Jul 30 14:48:16.614327 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.613059    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 14:48:16.614327 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.613849    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-n8tbb","kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal","openshift-cluster-node-tuning-operator/tuned-hb26z","openshift-dns/node-resolver-vhmks","openshift-image-registry/node-ca-x2w25","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal","openshift-multus/multus-additional-cni-plugins-dldgv","openshift-network-diagnostics/network-check-target-9tjwk","kube-system/konnectivity-agent-5gsdl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2","openshift-multus/multus-dwlts","openshift-multus/network-metrics-daemon-9vx62","openshift-network-operator/iptables-alerter-qnn6p","openshift-ovn-kubernetes/ovnkube-node-z46dq"]
Jul 30 14:48:16.616641 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.616609    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.619642 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.618669    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.619642 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.618919    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 14:48:16.619642 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.619151    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 14:48:16.619642 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.619174    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 14:48:16.619642 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.619287    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hpwxn\""
Jul 30 14:48:16.619642 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.619406    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 14:48:16.619642 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.619579    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 14:48:16.620380 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.620355    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 14:48:16.621521 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.620735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.621521 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.621130    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 14:48:16.621521 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.621325    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 14:48:16.621687 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.621534    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-b7j5d\""
Jul 30 14:48:16.622513 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.622381    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 14:48:16.622936 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.622918    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zdrxt\""
Jul 30 14:48:16.623016 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.622957    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 14:48:16.623492 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.623467    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.625931 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.625913    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 14:48:16.626013 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.625941    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lvf8r\""
Jul 30 14:48:16.626013 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.625999    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 14:48:16.627475 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.627259    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.629370 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.629345    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p97th\""
Jul 30 14:48:16.629552 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.629535    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 14:48:16.629711 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.629697    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 14:48:16.629911 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.629895    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 14:48:16.630659 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.630641    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.632737 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.632686    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:16.633753 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.633729    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 14:48:16.633841 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.633827    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tl2xx\""
Jul 30 14:48:16.633903 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.633873    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 14:48:16.633952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.633915    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 14:48:16.634031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.634018    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 14:48:16.634406 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.634362    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 14:48:16.634985 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.634968    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:16.635063 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.635035    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:16.635063 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.634817    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 14:48:16.635235 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.634747    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4cdzm\""
Jul 30 14:48:16.635495 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.634854    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 14:48:16.635811 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.634664    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 14:48:16.636364 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.636348    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 14:48:16.637083 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.637069    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:16.637162 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.637119    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:16.638994 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.638975    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.640731 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.640713    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 14:48:16.640899 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.640884    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 14:48:16.641023 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.641007    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 14:48:16.641073 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.641062    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.641428 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.641268    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jvgs6\""
Jul 30 14:48:16.643352 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.643254    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:16.643352 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.643323    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:16.643775 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.643748    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nggbp\""
Jul 30 14:48:16.643938 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.643919    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 14:48:16.647733 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.647702    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 14:48:16.648647 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.648627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-tuned\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.648740 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.648661    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19783e54-8469-4c0d-906d-f30863b87ea6-hosts-file\") pod \"node-resolver-vhmks\" (UID: \"19783e54-8469-4c0d-906d-f30863b87ea6\") " pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.648740 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.648685    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-run-netns\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.648740 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.648708    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-log-socket\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.648740 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.648728    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19783e54-8469-4c0d-906d-f30863b87ea6-tmp-dir\") pod \"node-resolver-vhmks\" (UID: \"19783e54-8469-4c0d-906d-f30863b87ea6\") " pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.649024 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.648751    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzxf2\" (UniqueName: \"kubernetes.io/projected/64206ee8-2cf7-42dd-9771-4ee52fd66a97-kube-api-access-zzxf2\") pod \"node-ca-x2w25\" (UID: \"64206ee8-2cf7-42dd-9771-4ee52fd66a97\") " pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.649024 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.648780    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc2hp\" (UniqueName: \"kubernetes.io/projected/627563ba-fde6-41e4-8f9a-98f7bb430783-kube-api-access-xc2hp\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.649257 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649232    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-run-systemd\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.649352 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-sysctl-conf\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.649352 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.649352 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c00a8c8a-3d2a-48d2-873e-b928edb51756-agent-certs\") pod \"konnectivity-agent-5gsdl\" (UID: \"c00a8c8a-3d2a-48d2-873e-b928edb51756\") " pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:16.649529 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649359    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-run\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.649529 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649380    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-os-release\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.649529 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649427    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c6fc3c7-6330-444e-b4e0-8bf15279f091-ovnkube-config\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.649529 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649448    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c6fc3c7-6330-444e-b4e0-8bf15279f091-ovn-node-metrics-cert\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.649529 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649468    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcg9l\" (UniqueName: \"kubernetes.io/projected/e4465ad0-7592-4811-9251-f0d3a533ec59-kube-api-access-bcg9l\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.649529 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649488    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-kubelet\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.649529 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649518    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c6fc3c7-6330-444e-b4e0-8bf15279f091-env-overrides\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649548    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-systemd\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649586    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-systemd-units\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649607    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-run-ovn-kubernetes\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649629    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-modprobe-d\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649648    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-host\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649666    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64206ee8-2cf7-42dd-9771-4ee52fd66a97-host\") pod \"node-ca-x2w25\" (UID: \"64206ee8-2cf7-42dd-9771-4ee52fd66a97\") " pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649685    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8680449d-85fb-485c-add0-e6d35a16f7a8-host-slash\") pod \"iptables-alerter-qnn6p\" (UID: \"8680449d-85fb-485c-add0-e6d35a16f7a8\") " pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649704    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-sysctl-d\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649729    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c6fc3c7-6330-444e-b4e0-8bf15279f091-ovnkube-script-lib\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649756    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/478b6274-9630-42e8-b363-e3e088f1bdbb-dbus\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649778    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-sys\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.649807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649802    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-var-lib-kubelet\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649822    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-run-ovn\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649845    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-cni-netd\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649893    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8680449d-85fb-485c-add0-e6d35a16f7a8-iptables-alerter-script\") pod \"iptables-alerter-qnn6p\" (UID: \"8680449d-85fb-485c-add0-e6d35a16f7a8\") " pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649915    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64206ee8-2cf7-42dd-9771-4ee52fd66a97-serviceca\") pod \"node-ca-x2w25\" (UID: \"64206ee8-2cf7-42dd-9771-4ee52fd66a97\") " pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649937    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/627563ba-fde6-41e4-8f9a-98f7bb430783-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649958    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8btq\" (UniqueName: \"kubernetes.io/projected/8680449d-85fb-485c-add0-e6d35a16f7a8-kube-api-access-g8btq\") pod \"iptables-alerter-qnn6p\" (UID: \"8680449d-85fb-485c-add0-e6d35a16f7a8\") " pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.649982    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-etc-openvswitch\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4465ad0-7592-4811-9251-f0d3a533ec59-tmp\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c00a8c8a-3d2a-48d2-873e-b928edb51756-konnectivity-ca\") pod \"konnectivity-agent-5gsdl\" (UID: \"c00a8c8a-3d2a-48d2-873e-b928edb51756\") " pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650183    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-slash\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650206    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-cni-bin\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.650272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650247    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650298    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-cnibin\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650326    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/627563ba-fde6-41e4-8f9a-98f7bb430783-cni-binary-copy\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650369    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-run-openvswitch\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650421    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-sysconfig\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650440    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-lib-modules\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650477    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l72g8\" (UniqueName: \"kubernetes.io/projected/19783e54-8469-4c0d-906d-f30863b87ea6-kube-api-access-l72g8\") pod \"node-resolver-vhmks\" (UID: \"19783e54-8469-4c0d-906d-f30863b87ea6\") " pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-var-lib-openvswitch\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650531    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/478b6274-9630-42e8-b363-e3e088f1bdbb-kubelet-config\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650560    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650581    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7z2g\" (UniqueName: \"kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g\") pod \"network-check-target-9tjwk\" (UID: \"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85\") " pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650615    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-system-cni-dir\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650646    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/627563ba-fde6-41e4-8f9a-98f7bb430783-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650678    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-node-log\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650715    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6c7v\" (UniqueName: \"kubernetes.io/projected/8c6fc3c7-6330-444e-b4e0-8bf15279f091-kube-api-access-w6c7v\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.650837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.650738    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-kubernetes\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.670784 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.670761    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qqjbj"
Jul 30 14:48:16.678731 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.678710    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qqjbj"
Jul 30 14:48:16.737582 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.737556    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 14:48:16.751286 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751258    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.751450 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c00a8c8a-3d2a-48d2-873e-b928edb51756-agent-certs\") pod \"konnectivity-agent-5gsdl\" (UID: \"c00a8c8a-3d2a-48d2-873e-b928edb51756\") " pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:16.751450 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751316    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-cnibin\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.751450 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751427    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-run\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.751606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751452    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-os-release\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.751606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751473    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c6fc3c7-6330-444e-b4e0-8bf15279f091-ovnkube-config\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.751606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751490    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.751606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751512    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c6fc3c7-6330-444e-b4e0-8bf15279f091-ovn-node-metrics-cert\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.751606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751536    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-run-netns\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.751606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751545    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-os-release\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.751606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751560    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcg9l\" (UniqueName: \"kubernetes.io/projected/e4465ad0-7592-4811-9251-f0d3a533ec59-kube-api-access-bcg9l\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.751606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-kubelet\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.751606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751589    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-run\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.751606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751610    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c6fc3c7-6330-444e-b4e0-8bf15279f091-env-overrides\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751630    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-device-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751653    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-systemd\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751677    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-systemd-units\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751698    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-run-ovn-kubernetes\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751729    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-var-lib-kubelet\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-modprobe-d\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751755    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751774    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-host\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64206ee8-2cf7-42dd-9771-4ee52fd66a97-host\") pod \"node-ca-x2w25\" (UID: \"64206ee8-2cf7-42dd-9771-4ee52fd66a97\") " pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-systemd\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751857    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8680449d-85fb-485c-add0-e6d35a16f7a8-host-slash\") pod \"iptables-alerter-qnn6p\" (UID: \"8680449d-85fb-485c-add0-e6d35a16f7a8\") " pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751873    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-kubelet\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751891    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-run-ovn-kubernetes\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751887    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-registration-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751920    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64206ee8-2cf7-42dd-9771-4ee52fd66a97-host\") pod \"node-ca-x2w25\" (UID: \"64206ee8-2cf7-42dd-9771-4ee52fd66a97\") " pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpw77\" (UniqueName: \"kubernetes.io/projected/6ed35c45-aaac-475b-be36-236cb83e7c89-kube-api-access-qpw77\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751963    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-os-release\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.752031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751968    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8680449d-85fb-485c-add0-e6d35a16f7a8-host-slash\") pod \"iptables-alerter-qnn6p\" (UID: \"8680449d-85fb-485c-add0-e6d35a16f7a8\") " pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.751986    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-hostroot\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-host\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752004    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-systemd-units\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752010    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-sysctl-d\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752055    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c6fc3c7-6330-444e-b4e0-8bf15279f091-ovnkube-script-lib\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752074    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/478b6274-9630-42e8-b363-e3e088f1bdbb-dbus\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752069    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c6fc3c7-6330-444e-b4e0-8bf15279f091-env-overrides\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c6fc3c7-6330-444e-b4e0-8bf15279f091-ovnkube-config\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752104    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-modprobe-d\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752106    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-daemon-config\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752137    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv4s9\" (UniqueName: \"kubernetes.io/projected/1b5481ac-491b-4bfb-ba36-8207c9449c6c-kube-api-access-hv4s9\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752128    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-sysctl-d\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752172    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752192    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-sys\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752210    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-var-lib-kubelet\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752230    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-run-ovn\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752270    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-cni-netd\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.752893 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752285    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-var-lib-kubelet\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752292    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-etc-kubernetes\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752288    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-run-ovn\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752287    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-sys\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752245    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/478b6274-9630-42e8-b363-e3e088f1bdbb-dbus\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752326    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-cni-netd\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8680449d-85fb-485c-add0-e6d35a16f7a8-iptables-alerter-script\") pod \"iptables-alerter-qnn6p\" (UID: \"8680449d-85fb-485c-add0-e6d35a16f7a8\") " pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752360    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-cni-dir\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64206ee8-2cf7-42dd-9771-4ee52fd66a97-serviceca\") pod \"node-ca-x2w25\" (UID: \"64206ee8-2cf7-42dd-9771-4ee52fd66a97\") " pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752458    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/627563ba-fde6-41e4-8f9a-98f7bb430783-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752483    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8btq\" (UniqueName: \"kubernetes.io/projected/8680449d-85fb-485c-add0-e6d35a16f7a8-kube-api-access-g8btq\") pod \"iptables-alerter-qnn6p\" (UID: \"8680449d-85fb-485c-add0-e6d35a16f7a8\") " pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752505    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-etc-openvswitch\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752527    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4465ad0-7592-4811-9251-f0d3a533ec59-tmp\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752546    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c00a8c8a-3d2a-48d2-873e-b928edb51756-konnectivity-ca\") pod \"konnectivity-agent-5gsdl\" (UID: \"c00a8c8a-3d2a-48d2-873e-b928edb51756\") " pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752546    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c6fc3c7-6330-444e-b4e0-8bf15279f091-ovnkube-script-lib\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752569    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-run-k8s-cni-cncf-io\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752591    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-var-lib-cni-multus\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.753681 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752606    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-etc-openvswitch\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-slash\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752667    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-cni-bin\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752692    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752717    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-system-cni-dir\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752786    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64206ee8-2cf7-42dd-9771-4ee52fd66a97-serviceca\") pod \"node-ca-x2w25\" (UID: \"64206ee8-2cf7-42dd-9771-4ee52fd66a97\") " pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-cni-bin\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752818    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8680449d-85fb-485c-add0-e6d35a16f7a8-iptables-alerter-script\") pod \"iptables-alerter-qnn6p\" (UID: \"8680449d-85fb-485c-add0-e6d35a16f7a8\") " pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752852    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-slash\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752892    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-socket-dir-parent\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752906    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752924    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-run-multus-certs\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.752978    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-cnibin\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/627563ba-fde6-41e4-8f9a-98f7bb430783-cni-binary-copy\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753114    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-cnibin\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-run-openvswitch\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753180    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-sys-fs\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.754500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-sysconfig\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753205    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c00a8c8a-3d2a-48d2-873e-b928edb51756-konnectivity-ca\") pod \"konnectivity-agent-5gsdl\" (UID: \"c00a8c8a-3d2a-48d2-873e-b928edb51756\") " pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753224    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-lib-modules\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753233    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-run-openvswitch\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753236    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/627563ba-fde6-41e4-8f9a-98f7bb430783-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l72g8\" (UniqueName: \"kubernetes.io/projected/19783e54-8469-4c0d-906d-f30863b87ea6-kube-api-access-l72g8\") pod \"node-resolver-vhmks\" (UID: \"19783e54-8469-4c0d-906d-f30863b87ea6\") " pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753271    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-var-lib-openvswitch\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753277    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-sysconfig\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/478b6274-9630-42e8-b363-e3e088f1bdbb-kubelet-config\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753316    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-var-lib-openvswitch\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753342    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-lib-modules\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753354    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/478b6274-9630-42e8-b363-e3e088f1bdbb-kubelet-config\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753348    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-socket-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753413    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7z2g\" (UniqueName: \"kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g\") pod \"network-check-target-9tjwk\" (UID: \"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85\") " pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753432    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-system-cni-dir\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.753448    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:16.755246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753455    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/627563ba-fde6-41e4-8f9a-98f7bb430783-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753473    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-node-log\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753500    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/627563ba-fde6-41e4-8f9a-98f7bb430783-system-cni-dir\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6c7v\" (UniqueName: \"kubernetes.io/projected/8c6fc3c7-6330-444e-b4e0-8bf15279f091-kube-api-access-w6c7v\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.753554    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret podName:478b6274-9630-42e8-b363-e3e088f1bdbb nodeName:}" failed. No retries permitted until 2026-07-30 14:48:17.2535315 +0000 UTC m=+2.926977612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret") pod "global-pull-secret-syncer-n8tbb" (UID: "478b6274-9630-42e8-b363-e3e088f1bdbb") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753578    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753608    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b5481ac-491b-4bfb-ba36-8207c9449c6c-cni-binary-copy\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753634    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-kubernetes\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753655    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-tuned\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753680    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19783e54-8469-4c0d-906d-f30863b87ea6-hosts-file\") pod \"node-resolver-vhmks\" (UID: \"19783e54-8469-4c0d-906d-f30863b87ea6\") " pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-run-netns\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753727    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-log-socket\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753750    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-etc-selinux\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753757    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-node-log\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753772    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-var-lib-cni-bin\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753794    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19783e54-8469-4c0d-906d-f30863b87ea6-tmp-dir\") pod \"node-resolver-vhmks\" (UID: \"19783e54-8469-4c0d-906d-f30863b87ea6\") " pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.755952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753814    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-kubernetes\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753818    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzxf2\" (UniqueName: \"kubernetes.io/projected/64206ee8-2cf7-42dd-9771-4ee52fd66a97-kube-api-access-zzxf2\") pod \"node-ca-x2w25\" (UID: \"64206ee8-2cf7-42dd-9771-4ee52fd66a97\") " pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753832    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-host-run-netns\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc2hp\" (UniqueName: \"kubernetes.io/projected/627563ba-fde6-41e4-8f9a-98f7bb430783-kube-api-access-xc2hp\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753877    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-run-systemd\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753915    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/627563ba-fde6-41e4-8f9a-98f7bb430783-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753900    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-conf-dir\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753972    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4phwx\" (UniqueName: \"kubernetes.io/projected/4be90577-11f7-4070-878c-7a6f277a401b-kube-api-access-4phwx\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.753999    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-sysctl-conf\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.754020    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-log-socket\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.754100    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/627563ba-fde6-41e4-8f9a-98f7bb430783-cni-binary-copy\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.754122    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19783e54-8469-4c0d-906d-f30863b87ea6-tmp-dir\") pod \"node-resolver-vhmks\" (UID: \"19783e54-8469-4c0d-906d-f30863b87ea6\") " pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.754143    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-sysctl-conf\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.754170    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19783e54-8469-4c0d-906d-f30863b87ea6-hosts-file\") pod \"node-resolver-vhmks\" (UID: \"19783e54-8469-4c0d-906d-f30863b87ea6\") " pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.754188    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c6fc3c7-6330-444e-b4e0-8bf15279f091-run-systemd\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.755492    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4465ad0-7592-4811-9251-f0d3a533ec59-tmp\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.755771    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c00a8c8a-3d2a-48d2-873e-b928edb51756-agent-certs\") pod \"konnectivity-agent-5gsdl\" (UID: \"c00a8c8a-3d2a-48d2-873e-b928edb51756\") " pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:16.756484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.755961    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c6fc3c7-6330-444e-b4e0-8bf15279f091-ovn-node-metrics-cert\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.756983 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.756314    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e4465ad0-7592-4811-9251-f0d3a533ec59-etc-tuned\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.760533 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.760515    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8btq\" (UniqueName: \"kubernetes.io/projected/8680449d-85fb-485c-add0-e6d35a16f7a8-kube-api-access-g8btq\") pod \"iptables-alerter-qnn6p\" (UID: \"8680449d-85fb-485c-add0-e6d35a16f7a8\") " pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.760883 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.760864    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcg9l\" (UniqueName: \"kubernetes.io/projected/e4465ad0-7592-4811-9251-f0d3a533ec59-kube-api-access-bcg9l\") pod \"tuned-hb26z\" (UID: \"e4465ad0-7592-4811-9251-f0d3a533ec59\") " pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.761889 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.761873    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:16.762127 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.761896    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:16.762127 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.761913    2582 projected.go:196] Error preparing data for projected volume kube-api-access-f7z2g for pod openshift-network-diagnostics/network-check-target-9tjwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:16.762127 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.761963    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g podName:2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:17.261949147 +0000 UTC m=+2.935395249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f7z2g" (UniqueName: "kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g") pod "network-check-target-9tjwk" (UID: "2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:16.763270 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.763245    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l72g8\" (UniqueName: \"kubernetes.io/projected/19783e54-8469-4c0d-906d-f30863b87ea6-kube-api-access-l72g8\") pod \"node-resolver-vhmks\" (UID: \"19783e54-8469-4c0d-906d-f30863b87ea6\") " pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.763708 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.763689    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6c7v\" (UniqueName: \"kubernetes.io/projected/8c6fc3c7-6330-444e-b4e0-8bf15279f091-kube-api-access-w6c7v\") pod \"ovnkube-node-z46dq\" (UID: \"8c6fc3c7-6330-444e-b4e0-8bf15279f091\") " pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.763932 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.763917    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc2hp\" (UniqueName: \"kubernetes.io/projected/627563ba-fde6-41e4-8f9a-98f7bb430783-kube-api-access-xc2hp\") pod \"multus-additional-cni-plugins-dldgv\" (UID: \"627563ba-fde6-41e4-8f9a-98f7bb430783\") " pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.764388 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.764341    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzxf2\" (UniqueName: \"kubernetes.io/projected/64206ee8-2cf7-42dd-9771-4ee52fd66a97-kube-api-access-zzxf2\") pod \"node-ca-x2w25\" (UID: \"64206ee8-2cf7-42dd-9771-4ee52fd66a97\") " pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.854951 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.854914    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-hostroot\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855115 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.854959    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-daemon-config\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855115 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.854983    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv4s9\" (UniqueName: \"kubernetes.io/projected/1b5481ac-491b-4bfb-ba36-8207c9449c6c-kube-api-access-hv4s9\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855115 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:16.855115 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855027    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-etc-kubernetes\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855115 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-cni-dir\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855115 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855072    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-run-k8s-cni-cncf-io\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855115 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855092    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-var-lib-cni-multus\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855115 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855116    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-system-cni-dir\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855137    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-socket-dir-parent\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855159    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-run-multus-certs\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-sys-fs\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855220    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-socket-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855254    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b5481ac-491b-4bfb-ba36-8207c9449c6c-cni-binary-copy\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855286    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-etc-selinux\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855307    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-var-lib-cni-bin\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855331    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-conf-dir\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855362    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4phwx\" (UniqueName: \"kubernetes.io/projected/4be90577-11f7-4070-878c-7a6f277a401b-kube-api-access-4phwx\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855411    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-cnibin\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855431    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-run-netns\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855467 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855449    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-device-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.855906 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855480    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-var-lib-kubelet\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855906 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-registration-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.855906 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855512    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpw77\" (UniqueName: \"kubernetes.io/projected/6ed35c45-aaac-475b-be36-236cb83e7c89-kube-api-access-qpw77\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.855906 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855531    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-os-release\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855906 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855625    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-os-release\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.855906 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.855668    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-hostroot\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856186 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856159    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-daemon-config\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856476    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-run-netns\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856492    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-var-lib-kubelet\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856513    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-run-k8s-cni-cncf-io\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856533    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-cnibin\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856561    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-etc-kubernetes\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856584    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-var-lib-cni-multus\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856583    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-device-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856582    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-registration-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856621    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-system-cni-dir\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856643    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-socket-dir-parent\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856648    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-sys-fs\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856672    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-conf-dir\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856733    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-etc-selinux\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856736    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ed35c45-aaac-475b-be36-236cb83e7c89-socket-dir\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856756    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-var-lib-cni-bin\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856773    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-host-run-multus-certs\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.856940 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856802    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b5481ac-491b-4bfb-ba36-8207c9449c6c-cni-binary-copy\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.857790 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.856820    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:16.857790 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:16.856861    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs podName:4be90577-11f7-4070-878c-7a6f277a401b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:17.35684911 +0000 UTC m=+3.030295199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs") pod "network-metrics-daemon-9vx62" (UID: "4be90577-11f7-4070-878c-7a6f277a401b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:16.857790 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.856820    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b5481ac-491b-4bfb-ba36-8207c9449c6c-multus-cni-dir\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.865037 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.865019    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpw77\" (UniqueName: \"kubernetes.io/projected/6ed35c45-aaac-475b-be36-236cb83e7c89-kube-api-access-qpw77\") pod \"aws-ebs-csi-driver-node-bhfm2\" (UID: \"6ed35c45-aaac-475b-be36-236cb83e7c89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.865780 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.865753    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv4s9\" (UniqueName: \"kubernetes.io/projected/1b5481ac-491b-4bfb-ba36-8207c9449c6c-kube-api-access-hv4s9\") pod \"multus-dwlts\" (UID: \"1b5481ac-491b-4bfb-ba36-8207c9449c6c\") " pod="openshift-multus/multus-dwlts"
Jul 30 14:48:16.865911 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.865895    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4phwx\" (UniqueName: \"kubernetes.io/projected/4be90577-11f7-4070-878c-7a6f277a401b-kube-api-access-4phwx\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:16.929836 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.929767    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dldgv"
Jul 30 14:48:16.945557 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.945525    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qnn6p"
Jul 30 14:48:16.954020 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.954000    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hb26z"
Jul 30 14:48:16.962383 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.962364    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vhmks"
Jul 30 14:48:16.971734 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.971714    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x2w25"
Jul 30 14:48:16.978074 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.978056    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:16.983449 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.983429    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:16.989611 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.989595    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2"
Jul 30 14:48:16.993984 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:16.993968    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dwlts"
Jul 30 14:48:17.258218 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.258156    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:17.258321 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:17.258293    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:17.258383 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:17.258371    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret podName:478b6274-9630-42e8-b363-e3e088f1bdbb nodeName:}" failed. No retries permitted until 2026-07-30 14:48:18.258344014 +0000 UTC m=+3.931790121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret") pod "global-pull-secret-syncer-n8tbb" (UID: "478b6274-9630-42e8-b363-e3e088f1bdbb") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:17.358948 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.358917    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:17.359103 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.358976    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7z2g\" (UniqueName: \"kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g\") pod \"network-check-target-9tjwk\" (UID: \"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85\") " pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:17.359103 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:17.359057    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:17.359202 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:17.359104    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:17.359202 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:17.359119    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:17.359202 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:17.359132    2582 projected.go:196] Error preparing data for projected volume kube-api-access-f7z2g for pod openshift-network-diagnostics/network-check-target-9tjwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:17.359202 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:17.359120    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs podName:4be90577-11f7-4070-878c-7a6f277a401b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:18.359106598 +0000 UTC m=+4.032552690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs") pod "network-metrics-daemon-9vx62" (UID: "4be90577-11f7-4070-878c-7a6f277a401b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:17.359202 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:17.359195    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g podName:2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:18.35917623 +0000 UTC m=+4.032622323 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7z2g" (UniqueName: "kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g") pod "network-check-target-9tjwk" (UID: "2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:17.513808 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:17.513783    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00a8c8a_3d2a_48d2_873e_b928edb51756.slice/crio-a5a93c518ce7232c1bd88ff70926f6b220d90bdcce91f3a8b1af622dd4ac10b4 WatchSource:0}: Error finding container a5a93c518ce7232c1bd88ff70926f6b220d90bdcce91f3a8b1af622dd4ac10b4: Status 404 returned error can't find the container with id a5a93c518ce7232c1bd88ff70926f6b220d90bdcce91f3a8b1af622dd4ac10b4
Jul 30 14:48:17.515439 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:17.515380    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6fc3c7_6330_444e_b4e0_8bf15279f091.slice/crio-7fb0574a604ff661fe260bd2df5779e75bd8023d18b4206f6a17610417ba75b2 WatchSource:0}: Error finding container 7fb0574a604ff661fe260bd2df5779e75bd8023d18b4206f6a17610417ba75b2: Status 404 returned error can't find the container with id 7fb0574a604ff661fe260bd2df5779e75bd8023d18b4206f6a17610417ba75b2
Jul 30 14:48:17.516778 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:17.516755    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8680449d_85fb_485c_add0_e6d35a16f7a8.slice/crio-ecf75b2b5e348086c42f13ce40e87c635934998d3be10cc6d0e522bb769f7b51 WatchSource:0}: Error finding container ecf75b2b5e348086c42f13ce40e87c635934998d3be10cc6d0e522bb769f7b51: Status 404 returned error can't find the container with id ecf75b2b5e348086c42f13ce40e87c635934998d3be10cc6d0e522bb769f7b51
Jul 30 14:48:17.517557 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:17.517539    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5481ac_491b_4bfb_ba36_8207c9449c6c.slice/crio-4e043b9c533bf4786db4b06b1eeda2ea0ab41c1bb77dfcc35605860a84a4a189 WatchSource:0}: Error finding container 4e043b9c533bf4786db4b06b1eeda2ea0ab41c1bb77dfcc35605860a84a4a189: Status 404 returned error can't find the container with id 4e043b9c533bf4786db4b06b1eeda2ea0ab41c1bb77dfcc35605860a84a4a189
Jul 30 14:48:17.519247 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:17.519227    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19783e54_8469_4c0d_906d_f30863b87ea6.slice/crio-372b2b03ff47e52c30845374647dcdce9dbbf259f405289b05f9828fbb7ccdbc WatchSource:0}: Error finding container 372b2b03ff47e52c30845374647dcdce9dbbf259f405289b05f9828fbb7ccdbc: Status 404 returned error can't find the container with id 372b2b03ff47e52c30845374647dcdce9dbbf259f405289b05f9828fbb7ccdbc
Jul 30 14:48:17.521211 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:17.521194    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64206ee8_2cf7_42dd_9771_4ee52fd66a97.slice/crio-b63ef467f20c0fea06c80a5a8fe6b3d064b4278c413431319349ef0e354dd59d WatchSource:0}: Error finding container b63ef467f20c0fea06c80a5a8fe6b3d064b4278c413431319349ef0e354dd59d: Status 404 returned error can't find the container with id b63ef467f20c0fea06c80a5a8fe6b3d064b4278c413431319349ef0e354dd59d
Jul 30 14:48:17.540713 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:17.540695    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed35c45_aaac_475b_be36_236cb83e7c89.slice/crio-3124db23d5664a711b2ca7539c40ab1f1e8c24faa1a5bc4e9ab5a72bade704ea WatchSource:0}: Error finding container 3124db23d5664a711b2ca7539c40ab1f1e8c24faa1a5bc4e9ab5a72bade704ea: Status 404 returned error can't find the container with id 3124db23d5664a711b2ca7539c40ab1f1e8c24faa1a5bc4e9ab5a72bade704ea
Jul 30 14:48:17.541991 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:17.541895    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4465ad0_7592_4811_9251_f0d3a533ec59.slice/crio-d4fea41bb9610030991831841df9fe276ba593361df084ef22017a7285f6809b WatchSource:0}: Error finding container d4fea41bb9610030991831841df9fe276ba593361df084ef22017a7285f6809b: Status 404 returned error can't find the container with id d4fea41bb9610030991831841df9fe276ba593361df084ef22017a7285f6809b
Jul 30 14:48:17.542595 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:17.542565    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod627563ba_fde6_41e4_8f9a_98f7bb430783.slice/crio-7b3075b352df44141b642ad556a7b90498d81e8275c9cfa899e472689600d49a WatchSource:0}: Error finding container 7b3075b352df44141b642ad556a7b90498d81e8275c9cfa899e472689600d49a: Status 404 returned error can't find the container with id 7b3075b352df44141b642ad556a7b90498d81e8275c9cfa899e472689600d49a
Jul 30 14:48:17.681954 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.681799    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 14:43:16 +0000 UTC" deadline="2028-03-05 20:09:32.997776642 +0000 UTC"
Jul 30 14:48:17.682321 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.681971    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14021h21m15.315823141s"
Jul 30 14:48:17.801932 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.801912    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:17.802020 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:17.801997    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:17.802061 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.802046    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:17.802137 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:17.802124    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:17.810137 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.810115    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x2w25" event={"ID":"64206ee8-2cf7-42dd-9771-4ee52fd66a97","Type":"ContainerStarted","Data":"b63ef467f20c0fea06c80a5a8fe6b3d064b4278c413431319349ef0e354dd59d"}
Jul 30 14:48:17.811281 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.811255    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dwlts" event={"ID":"1b5481ac-491b-4bfb-ba36-8207c9449c6c","Type":"ContainerStarted","Data":"4e043b9c533bf4786db4b06b1eeda2ea0ab41c1bb77dfcc35605860a84a4a189"}
Jul 30 14:48:17.812161 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.812145    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qnn6p" event={"ID":"8680449d-85fb-485c-add0-e6d35a16f7a8","Type":"ContainerStarted","Data":"ecf75b2b5e348086c42f13ce40e87c635934998d3be10cc6d0e522bb769f7b51"}
Jul 30 14:48:17.814031 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.814013    2582 generic.go:356] "Generic (PLEG): container finished" podID="cb369d9a4970133f5e28748fa24db29c" containerID="827577c2a2f1da598ecc91d18c6d6ecc85ff094eccc4327fff0fd3843de6ff05" exitCode=0
Jul 30 14:48:17.814113 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.814040    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal" event={"ID":"cb369d9a4970133f5e28748fa24db29c","Type":"ContainerDied","Data":"827577c2a2f1da598ecc91d18c6d6ecc85ff094eccc4327fff0fd3843de6ff05"}
Jul 30 14:48:17.815320 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.815297    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal" event={"ID":"958f474d6621921ad2d3c1a4ebe5848f","Type":"ContainerStarted","Data":"1c1691deb09cf8056b6b253b760b693485196eddc300f374423c7d1fe91e7092"}
Jul 30 14:48:17.816369 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.816353    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dldgv" event={"ID":"627563ba-fde6-41e4-8f9a-98f7bb430783","Type":"ContainerStarted","Data":"7b3075b352df44141b642ad556a7b90498d81e8275c9cfa899e472689600d49a"}
Jul 30 14:48:17.817429 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.817380    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vhmks" event={"ID":"19783e54-8469-4c0d-906d-f30863b87ea6","Type":"ContainerStarted","Data":"372b2b03ff47e52c30845374647dcdce9dbbf259f405289b05f9828fbb7ccdbc"}
Jul 30 14:48:17.818349 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.818330    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq" event={"ID":"8c6fc3c7-6330-444e-b4e0-8bf15279f091","Type":"ContainerStarted","Data":"7fb0574a604ff661fe260bd2df5779e75bd8023d18b4206f6a17610417ba75b2"}
Jul 30 14:48:17.819294 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.819278    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5gsdl" event={"ID":"c00a8c8a-3d2a-48d2-873e-b928edb51756","Type":"ContainerStarted","Data":"a5a93c518ce7232c1bd88ff70926f6b220d90bdcce91f3a8b1af622dd4ac10b4"}
Jul 30 14:48:17.820305 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.820289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hb26z" event={"ID":"e4465ad0-7592-4811-9251-f0d3a533ec59","Type":"ContainerStarted","Data":"d4fea41bb9610030991831841df9fe276ba593361df084ef22017a7285f6809b"}
Jul 30 14:48:17.821180 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.821164    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2" event={"ID":"6ed35c45-aaac-475b-be36-236cb83e7c89","Type":"ContainerStarted","Data":"3124db23d5664a711b2ca7539c40ab1f1e8c24faa1a5bc4e9ab5a72bade704ea"}
Jul 30 14:48:17.842134 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:17.842092    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-92.ec2.internal" podStartSLOduration=2.842078669 podStartE2EDuration="2.842078669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:17.841670624 +0000 UTC m=+3.515116734" watchObservedRunningTime="2026-07-30 14:48:17.842078669 +0000 UTC m=+3.515524779"
Jul 30 14:48:18.266243 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:18.265697    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:18.266243 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:18.265860    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:18.266243 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:18.265916    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret podName:478b6274-9630-42e8-b363-e3e088f1bdbb nodeName:}" failed. No retries permitted until 2026-07-30 14:48:20.265900002 +0000 UTC m=+5.939346100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret") pod "global-pull-secret-syncer-n8tbb" (UID: "478b6274-9630-42e8-b363-e3e088f1bdbb") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:18.366661 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:18.366631    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7z2g\" (UniqueName: \"kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g\") pod \"network-check-target-9tjwk\" (UID: \"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85\") " pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:18.366781 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:18.366693    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:18.366828 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:18.366795    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:18.366884 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:18.366862    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs podName:4be90577-11f7-4070-878c-7a6f277a401b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:20.36684507 +0000 UTC m=+6.040291180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs") pod "network-metrics-daemon-9vx62" (UID: "4be90577-11f7-4070-878c-7a6f277a401b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:18.367267 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:18.367245    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:18.367267 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:18.367266    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:18.367420 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:18.367278    2582 projected.go:196] Error preparing data for projected volume kube-api-access-f7z2g for pod openshift-network-diagnostics/network-check-target-9tjwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:18.376641 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:18.367630    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g podName:2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:20.367609807 +0000 UTC m=+6.041055943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7z2g" (UniqueName: "kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g") pod "network-check-target-9tjwk" (UID: "2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:18.682957 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:18.682872    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 14:43:16 +0000 UTC" deadline="2028-01-16 06:23:15.517571298 +0000 UTC"
Jul 30 14:48:18.682957 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:18.682929    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12831h34m56.834646726s"
Jul 30 14:48:18.802596 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:18.802513    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:18.802744 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:18.802665    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:18.839478 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:18.839433    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal" event={"ID":"cb369d9a4970133f5e28748fa24db29c","Type":"ContainerStarted","Data":"62da60f89e2c35ed2cd129da56e016a39663a2e7062709b6e79af4fae9b6b975"}
Jul 30 14:48:19.802610 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:19.801929    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:19.802610 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:19.802050    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:19.802610 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:19.802441    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:19.802610 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:19.802527    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:20.285727 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:20.285638    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:20.285894 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:20.285794    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:20.285894 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:20.285860    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret podName:478b6274-9630-42e8-b363-e3e088f1bdbb nodeName:}" failed. No retries permitted until 2026-07-30 14:48:24.285842152 +0000 UTC m=+9.959288259 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret") pod "global-pull-secret-syncer-n8tbb" (UID: "478b6274-9630-42e8-b363-e3e088f1bdbb") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:20.386218 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:20.386179    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:20.386413 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:20.386267    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7z2g\" (UniqueName: \"kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g\") pod \"network-check-target-9tjwk\" (UID: \"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85\") " pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:20.386480 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:20.386457    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:20.386539 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:20.386482    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:20.386539 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:20.386496    2582 projected.go:196] Error preparing data for projected volume kube-api-access-f7z2g for pod openshift-network-diagnostics/network-check-target-9tjwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:20.386641 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:20.386558    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g podName:2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:24.386537554 +0000 UTC m=+10.059983668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7z2g" (UniqueName: "kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g") pod "network-check-target-9tjwk" (UID: "2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:20.387008 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:20.386972    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:20.387115 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:20.387026    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs podName:4be90577-11f7-4070-878c-7a6f277a401b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:24.387009537 +0000 UTC m=+10.060455633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs") pod "network-metrics-daemon-9vx62" (UID: "4be90577-11f7-4070-878c-7a6f277a401b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:20.806317 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:20.806282    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:20.806772 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:20.806434    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:21.802286 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:21.802253    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:21.802485 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:21.802407    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:21.802485 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:21.802442    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:21.802620 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:21.802550    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:22.801595 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:22.801561    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:22.802035 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:22.801775    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:23.802587 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:23.801875    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:23.802587 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:23.801998    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:23.802587 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:23.802384    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:23.802587 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:23.802494    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:24.318704 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:24.318664    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:24.318878 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:24.318829    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:24.318956 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:24.318888    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret podName:478b6274-9630-42e8-b363-e3e088f1bdbb nodeName:}" failed. No retries permitted until 2026-07-30 14:48:32.318868643 +0000 UTC m=+17.992314739 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret") pod "global-pull-secret-syncer-n8tbb" (UID: "478b6274-9630-42e8-b363-e3e088f1bdbb") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:24.419196 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:24.419159    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7z2g\" (UniqueName: \"kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g\") pod \"network-check-target-9tjwk\" (UID: \"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85\") " pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:24.419362 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:24.419236    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:24.419362 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:24.419357    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:24.419517 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:24.419434    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs podName:4be90577-11f7-4070-878c-7a6f277a401b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:32.419414583 +0000 UTC m=+18.092860680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs") pod "network-metrics-daemon-9vx62" (UID: "4be90577-11f7-4070-878c-7a6f277a401b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:24.419790 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:24.419769    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:24.419790 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:24.419786    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:24.419935 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:24.419795    2582 projected.go:196] Error preparing data for projected volume kube-api-access-f7z2g for pod openshift-network-diagnostics/network-check-target-9tjwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:24.419935 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:24.419829    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g podName:2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:32.419817607 +0000 UTC m=+18.093263697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7z2g" (UniqueName: "kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g") pod "network-check-target-9tjwk" (UID: "2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:24.805511 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:24.804992    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:24.805511 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:24.805123    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:25.802050 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:25.801562    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:25.802050 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:25.801691    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:25.802050 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:25.801720    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:25.802050 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:25.801837    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:26.801937 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:26.801906    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:26.802389 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:26.802054    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:27.802358 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:27.802318    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:27.802749 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:27.802324    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:27.802749 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:27.802461    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:27.802749 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:27.802517    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:28.802069 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:28.802027    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:28.802233 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:28.802162    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:29.801929 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:29.801898    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:29.802384 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:29.801908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:29.802384 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:29.802013    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:29.802384 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:29.802095    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:30.801792 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:30.801756    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:30.801947 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:30.801907    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:31.802241 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:31.802213    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:31.802690 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:31.802302    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:31.802690 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:31.802355    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:31.802690 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:31.802451    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:32.384579 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:32.384545    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:32.384732 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:32.384689    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:32.384768 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:32.384733    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret podName:478b6274-9630-42e8-b363-e3e088f1bdbb nodeName:}" failed. No retries permitted until 2026-07-30 14:48:48.384719294 +0000 UTC m=+34.058165389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret") pod "global-pull-secret-syncer-n8tbb" (UID: "478b6274-9630-42e8-b363-e3e088f1bdbb") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:32.485359 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:32.485318    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7z2g\" (UniqueName: \"kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g\") pod \"network-check-target-9tjwk\" (UID: \"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85\") " pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:32.485562 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:32.485416    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:32.485630 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:32.485505    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:32.485630 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:32.485619    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:32.485723 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:32.485636    2582 projected.go:196] Error preparing data for projected volume kube-api-access-f7z2g for pod openshift-network-diagnostics/network-check-target-9tjwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:32.485723 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:32.485522    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:32.485723 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:32.485717    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g podName:2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:48.485696955 +0000 UTC m=+34.159143047 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7z2g" (UniqueName: "kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g") pod "network-check-target-9tjwk" (UID: "2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:32.485857 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:32.485755    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs podName:4be90577-11f7-4070-878c-7a6f277a401b nodeName:}" failed. No retries permitted until 2026-07-30 14:48:48.485733528 +0000 UTC m=+34.159179634 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs") pod "network-metrics-daemon-9vx62" (UID: "4be90577-11f7-4070-878c-7a6f277a401b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:32.805019 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:32.804947    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:32.805455 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:32.805070    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:33.801618 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:33.801577    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:33.801784 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:33.801577    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:33.801784 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:33.801715    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:33.801896 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:33.801780    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:34.803171 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:34.803135    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:34.803588 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:34.803263    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:35.802097 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:35.802064    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:35.802286 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:35.802069    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:35.802286 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:35.802184    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:35.802286 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:35.802249    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:36.802531 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.802101    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:36.803254 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:36.802647    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:36.878137 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.878070    2582 generic.go:356] "Generic (PLEG): container finished" podID="627563ba-fde6-41e4-8f9a-98f7bb430783" containerID="187c5c06e383220b200c06d68ee309c19fa72ed3e808d06becb3deee1baf8ec0" exitCode=0
Jul 30 14:48:36.878276 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.878143    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dldgv" event={"ID":"627563ba-fde6-41e4-8f9a-98f7bb430783","Type":"ContainerDied","Data":"187c5c06e383220b200c06d68ee309c19fa72ed3e808d06becb3deee1baf8ec0"}
Jul 30 14:48:36.880837 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.880814    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vhmks" event={"ID":"19783e54-8469-4c0d-906d-f30863b87ea6","Type":"ContainerStarted","Data":"d936b4dc95297c1c2135936e97437b4b2001981672105a623c4bb80fa15b089f"}
Jul 30 14:48:36.884306 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.884276    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq" event={"ID":"8c6fc3c7-6330-444e-b4e0-8bf15279f091","Type":"ContainerStarted","Data":"dd9c18dc7a1db52f2e8f16d6b5cac72e94d04bb18bdc8d5ff8eda25debf7fe40"}
Jul 30 14:48:36.884389 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.884335    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq" event={"ID":"8c6fc3c7-6330-444e-b4e0-8bf15279f091","Type":"ContainerStarted","Data":"6dc20f188869a257fba6d523c906e0a0f01ef90d8b55ae50550bde6de1a0d5e6"}
Jul 30 14:48:36.884389 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.884365    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq" event={"ID":"8c6fc3c7-6330-444e-b4e0-8bf15279f091","Type":"ContainerStarted","Data":"959386d3a1ba5f722fcf7300fe82db882d4e0d9da1b1f537494711f4d86605be"}
Jul 30 14:48:36.884497 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.884427    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq" event={"ID":"8c6fc3c7-6330-444e-b4e0-8bf15279f091","Type":"ContainerStarted","Data":"d4aac21f538198eb51eb5f4f8dee2f883287ecbf003f77153c100d4549e33424"}
Jul 30 14:48:36.884497 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.884456    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq" event={"ID":"8c6fc3c7-6330-444e-b4e0-8bf15279f091","Type":"ContainerStarted","Data":"8b543faec51fadb06b4c0a8d0dc51c04ed28dce72c29662fc9c2cca97336504c"}
Jul 30 14:48:36.884497 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.884488    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq" event={"ID":"8c6fc3c7-6330-444e-b4e0-8bf15279f091","Type":"ContainerStarted","Data":"6c946cc7f096ba44d73ce6710987db9362746bffcd966af0523e269906c9eca5"}
Jul 30 14:48:36.885594 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.885572    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5gsdl" event={"ID":"c00a8c8a-3d2a-48d2-873e-b928edb51756","Type":"ContainerStarted","Data":"61977bc336e17a3d3b38095d10c23e356076e1d031e1d53e76f152f020371d0a"}
Jul 30 14:48:36.887316 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.887285    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hb26z" event={"ID":"e4465ad0-7592-4811-9251-f0d3a533ec59","Type":"ContainerStarted","Data":"404a902dc833ed1403f582d93e12778319a634296beb67d32f4c9061b9f13b66"}
Jul 30 14:48:36.888504 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.888487    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2" event={"ID":"6ed35c45-aaac-475b-be36-236cb83e7c89","Type":"ContainerStarted","Data":"651fea13e673c42547389c25159efaf3372793532ab86d69b5fe61b3cf853a54"}
Jul 30 14:48:36.889733 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.889711    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x2w25" event={"ID":"64206ee8-2cf7-42dd-9771-4ee52fd66a97","Type":"ContainerStarted","Data":"8ed1e930efde913a475f935ad5da74af1142d5fe1a025b55d73f2cf2004fe75b"}
Jul 30 14:48:36.890917 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.890897    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dwlts" event={"ID":"1b5481ac-491b-4bfb-ba36-8207c9449c6c","Type":"ContainerStarted","Data":"943e8cc8b9263e7828d56e9992565a11568e2acc85154779569c821c366ecbb1"}
Jul 30 14:48:36.899162 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.899126    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-92.ec2.internal" podStartSLOduration=21.899115432 podStartE2EDuration="21.899115432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:18.854618867 +0000 UTC m=+4.528064979" watchObservedRunningTime="2026-07-30 14:48:36.899115432 +0000 UTC m=+22.572561545"
Jul 30 14:48:36.913726 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.913690    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hb26z" podStartSLOduration=4.502724581 podStartE2EDuration="22.913681642s" totalImagesPullingTime="18.410957061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:36.913623393 +0000 UTC m=+22.587069506" watchObservedRunningTime="2026-07-30 14:48:36.913681642 +0000 UTC m=+22.587127755"
Jul 30 14:48:36.927590 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.927547    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5gsdl" podStartSLOduration=4.50526876 podStartE2EDuration="22.927531601s" totalImagesPullingTime="18.422262841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:36.927246664 +0000 UTC m=+22.600692767" watchObservedRunningTime="2026-07-30 14:48:36.927531601 +0000 UTC m=+22.600977714"
Jul 30 14:48:36.941955 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.941923    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vhmks" podStartSLOduration=4.545214634 podStartE2EDuration="22.941910991s" totalImagesPullingTime="18.396696357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:36.941438401 +0000 UTC m=+22.614884514" watchObservedRunningTime="2026-07-30 14:48:36.941910991 +0000 UTC m=+22.615357104"
Jul 30 14:48:36.971218 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.971182    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dwlts" podStartSLOduration=4.496389114 podStartE2EDuration="22.971171366s" totalImagesPullingTime="18.474782252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:36.971115638 +0000 UTC m=+22.644561754" watchObservedRunningTime="2026-07-30 14:48:36.971171366 +0000 UTC m=+22.644617479"
Jul 30 14:48:36.971649 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:36.971597    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-x2w25" podStartSLOduration=4.57345474 podStartE2EDuration="22.97158855s" totalImagesPullingTime="18.39813381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:36.955429964 +0000 UTC m=+22.628876100" watchObservedRunningTime="2026-07-30 14:48:36.97158855 +0000 UTC m=+22.645034666"
Jul 30 14:48:37.114576 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:37.114544    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:37.200883 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:37.200859    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:37.384956 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:37.384928    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 14:48:37.713374 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:37.713204    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T14:48:37.384955207Z","UUID":"dc0b4e23-ab54-42b2-aa19-8adda5692050","Handler":null,"Name":"","Endpoint":""}
Jul 30 14:48:37.714992 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:37.714971    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 14:48:37.714992 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:37.715014    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 14:48:37.801619 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:37.801589    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:37.801765 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:37.801691    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:37.801833 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:37.801761    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:37.801914 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:37.801888    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:37.895716 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:37.895681    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qnn6p" event={"ID":"8680449d-85fb-485c-add0-e6d35a16f7a8","Type":"ContainerStarted","Data":"045c32fbf2f969325d3344ed8ebdbec69883a90c7fa2988e112780fb7ee58ab9"}
Jul 30 14:48:37.898651 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:37.898559    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2" event={"ID":"6ed35c45-aaac-475b-be36-236cb83e7c89","Type":"ContainerStarted","Data":"2fb42ad1813fd1d2af8b39c05f37e9e33b7430085deb57b76522167c89a8fbfa"}
Jul 30 14:48:38.029916 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:38.029836    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:38.084830 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:38.084797    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:38.148430 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:38.148383    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:38.803123 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:38.803058    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:38.803243 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:38.803196    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:38.905255 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:38.905154    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq" event={"ID":"8c6fc3c7-6330-444e-b4e0-8bf15279f091","Type":"ContainerStarted","Data":"c8fa4b95159150068add31317fba3976b51fb92e6c6006aba70b4cb97b6adbe4"}
Jul 30 14:48:38.909072 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:38.909041    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2" event={"ID":"6ed35c45-aaac-475b-be36-236cb83e7c89","Type":"ContainerStarted","Data":"3cfe6ae7e53560b5c44a5524e20f3aebf277ffa737b58219a8d369d4ad012006"}
Jul 30 14:48:38.926831 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:38.926790    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bhfm2" podStartSLOduration=4.261136929 podStartE2EDuration="24.926777214s" totalImagesPullingTime="20.665640285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:38.926466729 +0000 UTC m=+24.599912842" watchObservedRunningTime="2026-07-30 14:48:38.926777214 +0000 UTC m=+24.600223326"
Jul 30 14:48:38.926980 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:38.926962    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qnn6p" podStartSLOduration=6.503875247 podStartE2EDuration="24.926957671s" totalImagesPullingTime="18.423082424s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:37.911299302 +0000 UTC m=+23.584745426" watchObservedRunningTime="2026-07-30 14:48:38.926957671 +0000 UTC m=+24.600403783"
Jul 30 14:48:39.801751 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:39.801561    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:39.801931 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:39.801591    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:39.801931 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:39.801836    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:39.802032 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:39.801954    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:40.148296 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:40.148224    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:40.149034 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:40.149013    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:40.801690 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:40.801653    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:40.801869 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:40.801793    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:41.374363 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:41.374343    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:41.374864 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:41.374846    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5gsdl"
Jul 30 14:48:41.436254 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:41.436229    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:41.801886 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:41.801726    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:41.801962 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:41.801726    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:41.801995 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:41.801961    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:41.802055 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:41.802035    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:41.918023 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:41.917974    2582 generic.go:356] "Generic (PLEG): container finished" podID="627563ba-fde6-41e4-8f9a-98f7bb430783" containerID="03d14344afaa1b5db03e187da58d285140322d1aeab853cd358477a2aeaeb4a3" exitCode=0
Jul 30 14:48:41.918240 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:41.918052    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dldgv" event={"ID":"627563ba-fde6-41e4-8f9a-98f7bb430783","Type":"ContainerDied","Data":"03d14344afaa1b5db03e187da58d285140322d1aeab853cd358477a2aeaeb4a3"}
Jul 30 14:48:41.921890 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:41.921867    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq" event={"ID":"8c6fc3c7-6330-444e-b4e0-8bf15279f091","Type":"ContainerStarted","Data":"192a51d2d4c8d14194348f6361f59b25d20134fc2776ad158428a3bb08764c97"}
Jul 30 14:48:41.961011 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:41.960973    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq" podStartSLOduration=9.474508129 podStartE2EDuration="27.960961157s" totalImagesPullingTime="18.486453028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:41.960899666 +0000 UTC m=+27.634345779" watchObservedRunningTime="2026-07-30 14:48:41.960961157 +0000 UTC m=+27.634407270"
Jul 30 14:48:42.801930 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:42.801895    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:42.802665 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:42.802050    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:42.925029 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:42.924998    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:42.925261 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:42.925080    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:42.925261 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:42.925109    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:42.940622 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:42.940593    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:42.941496 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:42.941477    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:48:43.394000 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:43.393969    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n8tbb"]
Jul 30 14:48:43.394153 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:43.394138    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:43.394276 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:43.394254    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:43.397160 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:43.397136    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9tjwk"]
Jul 30 14:48:43.397260 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:43.397249    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:43.397437 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:43.397374    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:43.397830 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:43.397801    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9vx62"]
Jul 30 14:48:43.397929 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:43.397915    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:43.398043 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:43.398024    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:43.928175 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:43.928142    2582 generic.go:356] "Generic (PLEG): container finished" podID="627563ba-fde6-41e4-8f9a-98f7bb430783" containerID="bb086f6f264ea546c2f6e793012296333edee95ab9e9f75aa32639ac593ec909" exitCode=0
Jul 30 14:48:43.928543 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:43.928225    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dldgv" event={"ID":"627563ba-fde6-41e4-8f9a-98f7bb430783","Type":"ContainerDied","Data":"bb086f6f264ea546c2f6e793012296333edee95ab9e9f75aa32639ac593ec909"}
Jul 30 14:48:44.426390 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:44.426316    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:44.498501 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:44.498468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:44.805046 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:44.804831    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:44.805167 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:44.804849    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:44.805167 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:44.805093    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:44.805167 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:44.804881    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:44.805167 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:44.805158    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:44.805297 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:44.805219    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:45.934855 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:45.934772    2582 generic.go:356] "Generic (PLEG): container finished" podID="627563ba-fde6-41e4-8f9a-98f7bb430783" containerID="c347502718e81b7a75c0c075c11695bdacf92a611ffb977c0a9e142b5c51d99e" exitCode=0
Jul 30 14:48:45.934855 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:45.934831    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dldgv" event={"ID":"627563ba-fde6-41e4-8f9a-98f7bb430783","Type":"ContainerDied","Data":"c347502718e81b7a75c0c075c11695bdacf92a611ffb977c0a9e142b5c51d99e"}
Jul 30 14:48:46.801562 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:46.801531    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:46.801735 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:46.801634    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tjwk" podUID="2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85"
Jul 30 14:48:46.802008 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:46.801989    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:46.802109 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:46.802089    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8tbb" podUID="478b6274-9630-42e8-b363-e3e088f1bdbb"
Jul 30 14:48:46.802151 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:46.802134    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:46.802197 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:46.802185    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9vx62" podUID="4be90577-11f7-4070-878c-7a6f277a401b"
Jul 30 14:48:48.406366 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.406323    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:48.406872 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:48.406522    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:48.406872 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:48.406585    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret podName:478b6274-9630-42e8-b363-e3e088f1bdbb nodeName:}" failed. No retries permitted until 2026-07-30 14:49:20.406568315 +0000 UTC m=+66.080014428 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret") pod "global-pull-secret-syncer-n8tbb" (UID: "478b6274-9630-42e8-b363-e3e088f1bdbb") : object "kube-system"/"original-pull-secret" not registered
Jul 30 14:48:48.506963 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.506932    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7z2g\" (UniqueName: \"kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g\") pod \"network-check-target-9tjwk\" (UID: \"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85\") " pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:48.507087 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.506990    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:48.507133 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:48.507098    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:48.507182 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:48.507148    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs podName:4be90577-11f7-4070-878c-7a6f277a401b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:20.507133303 +0000 UTC m=+66.180579395 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs") pod "network-metrics-daemon-9vx62" (UID: "4be90577-11f7-4070-878c-7a6f277a401b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 14:48:48.507239 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:48.507226    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 14:48:48.507289 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:48.507246    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 14:48:48.507289 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:48.507258    2582 projected.go:196] Error preparing data for projected volume kube-api-access-f7z2g for pod openshift-network-diagnostics/network-check-target-9tjwk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:48.507364 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:48.507294    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g podName:2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:20.507282046 +0000 UTC m=+66.180728140 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f7z2g" (UniqueName: "kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g") pod "network-check-target-9tjwk" (UID: "2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 14:48:48.630668 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.630627    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:48.647377 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.647351    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-92.ec2.internal" event="NodeReady"
Jul 30 14:48:48.647558 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.647518    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 14:48:48.691494 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.691469    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jjcpx"]
Jul 30 14:48:48.696547 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.696528    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rzcsk"]
Jul 30 14:48:48.696690 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.696672    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.698050 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.698022    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:48.699176 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.699017    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 14:48:48.699176 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.699060    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 14:48:48.699176 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.699062    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vrbw2\""
Jul 30 14:48:48.699834 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.699814    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rzcsk"
Jul 30 14:48:48.701655 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.701635    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 14:48:48.701773 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.701689    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dcx9h\""
Jul 30 14:48:48.701773 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.701749    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 14:48:48.701885 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.701831    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 14:48:48.705723 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.705706    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jjcpx"]
Jul 30 14:48:48.707489 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.707471    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rzcsk"]
Jul 30 14:48:48.801872 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.801839    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:48:48.802023 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.801839    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:48:48.802023 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.801851    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:48:48.804020 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.803996    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 14:48:48.804546 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.804528    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dd6sm\""
Jul 30 14:48:48.804639 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.804528    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-82hs8\""
Jul 30 14:48:48.804639 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.804612    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 14:48:48.804917 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.804892    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 14:48:48.805007 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.804986    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 14:48:48.808999 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.808977    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8vtd\" (UniqueName: \"kubernetes.io/projected/cb36fd80-2c52-40fd-b9bf-4868a44b4030-kube-api-access-l8vtd\") pod \"ingress-canary-rzcsk\" (UID: \"cb36fd80-2c52-40fd-b9bf-4868a44b4030\") " pod="openshift-ingress-canary/ingress-canary-rzcsk"
Jul 30 14:48:48.809093 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.809039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-config-volume\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.809093 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.809069    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb36fd80-2c52-40fd-b9bf-4868a44b4030-cert\") pod \"ingress-canary-rzcsk\" (UID: \"cb36fd80-2c52-40fd-b9bf-4868a44b4030\") " pod="openshift-ingress-canary/ingress-canary-rzcsk"
Jul 30 14:48:48.809193 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.809095    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-tmp-dir\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.809193 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.809119    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjhdn\" (UniqueName: \"kubernetes.io/projected/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-kube-api-access-tjhdn\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.809193 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.809183    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.822841 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.821913    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:48.882803 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.882768    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:48.909611 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.909585    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb36fd80-2c52-40fd-b9bf-4868a44b4030-cert\") pod \"ingress-canary-rzcsk\" (UID: \"cb36fd80-2c52-40fd-b9bf-4868a44b4030\") " pod="openshift-ingress-canary/ingress-canary-rzcsk"
Jul 30 14:48:48.909728 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.909613    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-tmp-dir\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.909728 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.909632    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjhdn\" (UniqueName: \"kubernetes.io/projected/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-kube-api-access-tjhdn\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.909728 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.909702    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.909886 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.909737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8vtd\" (UniqueName: \"kubernetes.io/projected/cb36fd80-2c52-40fd-b9bf-4868a44b4030-kube-api-access-l8vtd\") pod \"ingress-canary-rzcsk\" (UID: \"cb36fd80-2c52-40fd-b9bf-4868a44b4030\") " pod="openshift-ingress-canary/ingress-canary-rzcsk"
Jul 30 14:48:48.909886 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.909784    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-config-volume\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.909987 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.909972    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-tmp-dir\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.910069 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:48.910045    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:48:48.910187 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:48.910109    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls podName:e7cd0a49-d9f2-42d7-9f3a-abc723a0d944 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:49.410091038 +0000 UTC m=+35.083537141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls") pod "dns-default-jjcpx" (UID: "e7cd0a49-d9f2-42d7-9f3a-abc723a0d944") : secret "dns-default-metrics-tls" not found
Jul 30 14:48:48.910565 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.910523    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-config-volume\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.914244 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.914224    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb36fd80-2c52-40fd-b9bf-4868a44b4030-cert\") pod \"ingress-canary-rzcsk\" (UID: \"cb36fd80-2c52-40fd-b9bf-4868a44b4030\") " pod="openshift-ingress-canary/ingress-canary-rzcsk"
Jul 30 14:48:48.930860 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.930842    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjhdn\" (UniqueName: \"kubernetes.io/projected/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-kube-api-access-tjhdn\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:48.932879 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:48.932857    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8vtd\" (UniqueName: \"kubernetes.io/projected/cb36fd80-2c52-40fd-b9bf-4868a44b4030-kube-api-access-l8vtd\") pod \"ingress-canary-rzcsk\" (UID: \"cb36fd80-2c52-40fd-b9bf-4868a44b4030\") " pod="openshift-ingress-canary/ingress-canary-rzcsk"
Jul 30 14:48:49.016543 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:49.016465    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rzcsk"
Jul 30 14:48:49.302011 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:49.301907    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:49.362778 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:49.362743    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:49.413756 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:49.413717    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:49.414471 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:49.413883    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:48:49.414471 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:49.413960    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls podName:e7cd0a49-d9f2-42d7-9f3a-abc723a0d944 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:50.413938946 +0000 UTC m=+36.087385037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls") pod "dns-default-jjcpx" (UID: "e7cd0a49-d9f2-42d7-9f3a-abc723a0d944") : secret "dns-default-metrics-tls" not found
Jul 30 14:48:50.241345 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:50.241146    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rzcsk"]
Jul 30 14:48:50.248345 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:48:50.248323    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb36fd80_2c52_40fd_b9bf_4868a44b4030.slice/crio-8d1a102b0702f6e2ab4a8cc589c6f211a1098acf23d1ab0b13845e4534128f98 WatchSource:0}: Error finding container 8d1a102b0702f6e2ab4a8cc589c6f211a1098acf23d1ab0b13845e4534128f98: Status 404 returned error can't find the container with id 8d1a102b0702f6e2ab4a8cc589c6f211a1098acf23d1ab0b13845e4534128f98
Jul 30 14:48:50.420891 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:50.420865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:50.421177 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:50.421014    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:48:50.421177 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:50.421078    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls podName:e7cd0a49-d9f2-42d7-9f3a-abc723a0d944 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:52.421062454 +0000 UTC m=+38.094508547 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls") pod "dns-default-jjcpx" (UID: "e7cd0a49-d9f2-42d7-9f3a-abc723a0d944") : secret "dns-default-metrics-tls" not found
Jul 30 14:48:50.949414 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:50.949361    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rzcsk" event={"ID":"cb36fd80-2c52-40fd-b9bf-4868a44b4030","Type":"ContainerStarted","Data":"8d1a102b0702f6e2ab4a8cc589c6f211a1098acf23d1ab0b13845e4534128f98"}
Jul 30 14:48:50.952429 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:50.952385    2582 generic.go:356] "Generic (PLEG): container finished" podID="627563ba-fde6-41e4-8f9a-98f7bb430783" containerID="157fe9bddc3ecf46d2e61e1f519f4a66e314a4d72d4ab0e67b513d1704988066" exitCode=0
Jul 30 14:48:50.952561 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:50.952452    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dldgv" event={"ID":"627563ba-fde6-41e4-8f9a-98f7bb430783","Type":"ContainerDied","Data":"157fe9bddc3ecf46d2e61e1f519f4a66e314a4d72d4ab0e67b513d1704988066"}
Jul 30 14:48:51.959243 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:51.959204    2582 generic.go:356] "Generic (PLEG): container finished" podID="627563ba-fde6-41e4-8f9a-98f7bb430783" containerID="6f703b4f4abf81afd965759dc97bb368de65a0899b27a9a00339cfed133702ad" exitCode=0
Jul 30 14:48:51.959767 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:51.959289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dldgv" event={"ID":"627563ba-fde6-41e4-8f9a-98f7bb430783","Type":"ContainerDied","Data":"6f703b4f4abf81afd965759dc97bb368de65a0899b27a9a00339cfed133702ad"}
Jul 30 14:48:52.436791 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:52.436748    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:52.436927 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:52.436899    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:48:52.436979 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:52.436964    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls podName:e7cd0a49-d9f2-42d7-9f3a-abc723a0d944 nodeName:}" failed. No retries permitted until 2026-07-30 14:48:56.436948824 +0000 UTC m=+42.110394918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls") pod "dns-default-jjcpx" (UID: "e7cd0a49-d9f2-42d7-9f3a-abc723a0d944") : secret "dns-default-metrics-tls" not found
Jul 30 14:48:52.967008 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:52.966856    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dldgv" event={"ID":"627563ba-fde6-41e4-8f9a-98f7bb430783","Type":"ContainerStarted","Data":"359bc749976e41b03c21964bcf141569a1bf3f32858d3a80dd13a1e6ae518fee"}
Jul 30 14:48:52.990290 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:52.990248    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dldgv" podStartSLOduration=9.360584845 podStartE2EDuration="38.990229889s" totalImagesPullingTime="29.629645044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:52.989428063 +0000 UTC m=+38.662874176" watchObservedRunningTime="2026-07-30 14:48:52.990229889 +0000 UTC m=+38.663676001"
Jul 30 14:48:53.117140 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:53.117096    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:53.188970 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:53.188939    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:53.319831 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:53.319791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:53.417949 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:53.417919    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:53.876378 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:53.876345    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:53.970540 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:53.970511    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rzcsk" event={"ID":"cb36fd80-2c52-40fd-b9bf-4868a44b4030","Type":"ContainerStarted","Data":"716351c7ab543f69f41df7ecead69d86900fba4ce1e738967690c585ad96ecb1"}
Jul 30 14:48:53.976631 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:53.976607    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:53.987591 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:53.987546    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rzcsk" podStartSLOduration=3.36781838 podStartE2EDuration="5.987532263s" totalImagesPullingTime="2.619713883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:48:53.987472287 +0000 UTC m=+39.660918399" watchObservedRunningTime="2026-07-30 14:48:53.987532263 +0000 UTC m=+39.660978376"
Jul 30 14:48:54.077442 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:54.077418    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:48:56.467533 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:48:56.467500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:48:56.467909 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:56.467652    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:48:56.467909 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:48:56.467718    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls podName:e7cd0a49-d9f2-42d7-9f3a-abc723a0d944 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:04.467702705 +0000 UTC m=+50.141148797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls") pod "dns-default-jjcpx" (UID: "e7cd0a49-d9f2-42d7-9f3a-abc723a0d944") : secret "dns-default-metrics-tls" not found
Jul 30 14:49:01.054465 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.054436    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx"]
Jul 30 14:49:01.097180 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.097150    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-76489d7779-qr8pc"]
Jul 30 14:49:01.097342 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.097290    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx"
Jul 30 14:49:01.099278 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.099255    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 30 14:49:01.099413 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.099310    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 30 14:49:01.099413 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.099265    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-rnq6g\""
Jul 30 14:49:01.110726 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.110704    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx"]
Jul 30 14:49:01.110829 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.110740    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76489d7779-qr8pc"]
Jul 30 14:49:01.110829 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.110817    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.113124 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.112953    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 14:49:01.113124 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.112993    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 30 14:49:01.113124 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.113007    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 30 14:49:01.113124 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.113015    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 30 14:49:01.113124 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.113073    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 30 14:49:01.113124 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.113087    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jpgsk\""
Jul 30 14:49:01.113124 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.113018    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 14:49:01.153458 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.153438    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"]
Jul 30 14:49:01.170613 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.168565    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"]
Jul 30 14:49:01.171634 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.171613    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:01.175330 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.175301    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 30 14:49:01.175625 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.175610    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hdgfm\""
Jul 30 14:49:01.175701 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.175613    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 30 14:49:01.192492 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.192469    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"]
Jul 30 14:49:01.192578 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.192496    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"]
Jul 30 14:49:01.192578 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.192557    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.194747 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.194648    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 30 14:49:01.195444 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.195425    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-lfswv\""
Jul 30 14:49:01.195544 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.195517    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 30 14:49:01.195744 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.195730    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 30 14:49:01.196445 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.196428    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 30 14:49:01.199210 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.199193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.199274 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.199219    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.199274 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.199244    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vskgl\" (UniqueName: \"kubernetes.io/projected/bea29110-940e-491a-844b-966b31f8dea2-kube-api-access-vskgl\") pod \"volume-data-source-validator-549c6d54cf-8z9mx\" (UID: \"bea29110-940e-491a-844b-966b31f8dea2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx"
Jul 30 14:49:01.199344 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.199287    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-stats-auth\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.199377 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.199362    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-default-certificate\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.199433 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.199379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ntbm\" (UniqueName: \"kubernetes.io/projected/f31dcb58-4fd7-425b-b5a8-51dfac54437f-kube-api-access-9ntbm\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.239482 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.239454    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:01.300488 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300464    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.300600 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300497    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:01.300600 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300517    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29r5r\" (UniqueName: \"kubernetes.io/projected/2b9a1df1-f0ea-4efb-b71d-ab8a20761180-kube-api-access-29r5r\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql\" (UID: \"2b9a1df1-f0ea-4efb-b71d-ab8a20761180\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.300600 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300563    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vskgl\" (UniqueName: \"kubernetes.io/projected/bea29110-940e-491a-844b-966b31f8dea2-kube-api-access-vskgl\") pod \"volume-data-source-validator-549c6d54cf-8z9mx\" (UID: \"bea29110-940e-491a-844b-966b31f8dea2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx"
Jul 30 14:49:01.300600 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300597    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-stats-auth\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.300804 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:01.300607    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 14:49:01.300804 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:01.300649    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:01.800635175 +0000 UTC m=+47.474081265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : secret "router-metrics-certs-default" not found
Jul 30 14:49:01.300804 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:01.300804 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-default-certificate\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.300804 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300748    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ntbm\" (UniqueName: \"kubernetes.io/projected/f31dcb58-4fd7-425b-b5a8-51dfac54437f-kube-api-access-9ntbm\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.301018 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300870    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b9a1df1-f0ea-4efb-b71d-ab8a20761180-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql\" (UID: \"2b9a1df1-f0ea-4efb-b71d-ab8a20761180\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.301018 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300900    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.301018 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.300922    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b9a1df1-f0ea-4efb-b71d-ab8a20761180-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql\" (UID: \"2b9a1df1-f0ea-4efb-b71d-ab8a20761180\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.301118 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:01.301022    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:01.801010403 +0000 UTC m=+47.474456494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : configmap references non-existent config key: service-ca.crt
Jul 30 14:49:01.304742 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.304694    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-default-certificate\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.304742 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.304701    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-stats-auth\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.308600 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.308578    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ntbm\" (UniqueName: \"kubernetes.io/projected/f31dcb58-4fd7-425b-b5a8-51dfac54437f-kube-api-access-9ntbm\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.309014 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.308995    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vskgl\" (UniqueName: \"kubernetes.io/projected/bea29110-940e-491a-844b-966b31f8dea2-kube-api-access-vskgl\") pod \"volume-data-source-validator-549c6d54cf-8z9mx\" (UID: \"bea29110-940e-491a-844b-966b31f8dea2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx"
Jul 30 14:49:01.344066 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.344035    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:01.402161 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.402138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:01.402242 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.402184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b9a1df1-f0ea-4efb-b71d-ab8a20761180-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql\" (UID: \"2b9a1df1-f0ea-4efb-b71d-ab8a20761180\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.402242 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.402210    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b9a1df1-f0ea-4efb-b71d-ab8a20761180-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql\" (UID: \"2b9a1df1-f0ea-4efb-b71d-ab8a20761180\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.402319 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.402249    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:01.402319 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:01.402261    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 14:49:01.402319 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.402272    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29r5r\" (UniqueName: \"kubernetes.io/projected/2b9a1df1-f0ea-4efb-b71d-ab8a20761180-kube-api-access-29r5r\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql\" (UID: \"2b9a1df1-f0ea-4efb-b71d-ab8a20761180\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.402468 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:01.402324    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert podName:f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce nodeName:}" failed. No retries permitted until 2026-07-30 14:49:01.902307822 +0000 UTC m=+47.575753933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-jbpq8" (UID: "f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce") : secret "networking-console-plugin-cert" not found
Jul 30 14:49:01.402738 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.402715    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2b9a1df1-f0ea-4efb-b71d-ab8a20761180-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql\" (UID: \"2b9a1df1-f0ea-4efb-b71d-ab8a20761180\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.402890 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.402873    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:01.404289 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.404270    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2b9a1df1-f0ea-4efb-b71d-ab8a20761180-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql\" (UID: \"2b9a1df1-f0ea-4efb-b71d-ab8a20761180\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.406288 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.406270    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx"
Jul 30 14:49:01.410461 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.410442    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29r5r\" (UniqueName: \"kubernetes.io/projected/2b9a1df1-f0ea-4efb-b71d-ab8a20761180-kube-api-access-29r5r\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql\" (UID: \"2b9a1df1-f0ea-4efb-b71d-ab8a20761180\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.501475 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.501444    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"
Jul 30 14:49:01.553757 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.553728    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx"]
Jul 30 14:49:01.556731 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:01.556697    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbea29110_940e_491a_844b_966b31f8dea2.slice/crio-c44b0e444ce7ff956762db4f8f9faa4e911a73ce7facee5a3b8c1e11031b4342 WatchSource:0}: Error finding container c44b0e444ce7ff956762db4f8f9faa4e911a73ce7facee5a3b8c1e11031b4342: Status 404 returned error can't find the container with id c44b0e444ce7ff956762db4f8f9faa4e911a73ce7facee5a3b8c1e11031b4342
Jul 30 14:49:01.654656 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.654625    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql"]
Jul 30 14:49:01.659266 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:01.659242    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9a1df1_f0ea_4efb_b71d_ab8a20761180.slice/crio-b666bb38269e9b8cbcf5b4ae482edb32b6d773151affa0446100320ba7d20e71 WatchSource:0}: Error finding container b666bb38269e9b8cbcf5b4ae482edb32b6d773151affa0446100320ba7d20e71: Status 404 returned error can't find the container with id b666bb38269e9b8cbcf5b4ae482edb32b6d773151affa0446100320ba7d20e71
Jul 30 14:49:01.806242 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.806148    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.806242 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.806193    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:01.806469 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:01.806299    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 14:49:01.806469 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:01.806335    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:02.806314539 +0000 UTC m=+48.479760635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : configmap references non-existent config key: service-ca.crt
Jul 30 14:49:01.806469 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:01.806361    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:02.806354548 +0000 UTC m=+48.479800640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : secret "router-metrics-certs-default" not found
Jul 30 14:49:01.907119 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.907082    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:01.907285 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:01.907208    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 14:49:01.907285 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:01.907271    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert podName:f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce nodeName:}" failed. No retries permitted until 2026-07-30 14:49:02.907253841 +0000 UTC m=+48.580699944 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-jbpq8" (UID: "f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce") : secret "networking-console-plugin-cert" not found
Jul 30 14:49:01.990441 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.990388    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql" event={"ID":"2b9a1df1-f0ea-4efb-b71d-ab8a20761180","Type":"ContainerStarted","Data":"b666bb38269e9b8cbcf5b4ae482edb32b6d773151affa0446100320ba7d20e71"}
Jul 30 14:49:01.991464 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:01.991447    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx" event={"ID":"bea29110-940e-491a-844b-966b31f8dea2","Type":"ContainerStarted","Data":"c44b0e444ce7ff956762db4f8f9faa4e911a73ce7facee5a3b8c1e11031b4342"}
Jul 30 14:49:02.812604 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:02.812568    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:02.813034 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:02.812621    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:02.813034 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:02.812775    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:04.812751885 +0000 UTC m=+50.486197981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : configmap references non-existent config key: service-ca.crt
Jul 30 14:49:02.813034 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:02.812784    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 14:49:02.813034 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:02.812853    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:04.812836742 +0000 UTC m=+50.486282840 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : secret "router-metrics-certs-default" not found
Jul 30 14:49:02.913928 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:02.913893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:02.914119 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:02.914067    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 14:49:02.914180 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:02.914124    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert podName:f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce nodeName:}" failed. No retries permitted until 2026-07-30 14:49:04.914110598 +0000 UTC m=+50.587556694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-jbpq8" (UID: "f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce") : secret "networking-console-plugin-cert" not found
Jul 30 14:49:04.526346 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:04.526308    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:49:04.526725 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:04.526485    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 14:49:04.526725 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:04.526563    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls podName:e7cd0a49-d9f2-42d7-9f3a-abc723a0d944 nodeName:}" failed. No retries permitted until 2026-07-30 14:49:20.526542916 +0000 UTC m=+66.199989008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls") pod "dns-default-jjcpx" (UID: "e7cd0a49-d9f2-42d7-9f3a-abc723a0d944") : secret "dns-default-metrics-tls" not found
Jul 30 14:49:04.828790 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:04.828695    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:04.828790 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:04.828747    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:04.828973 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:04.828897    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:08.828878093 +0000 UTC m=+54.502324184 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : configmap references non-existent config key: service-ca.crt
Jul 30 14:49:04.828973 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:04.828909    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 14:49:04.828973 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:04.828959    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:08.828946209 +0000 UTC m=+54.502392301 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : secret "router-metrics-certs-default" not found
Jul 30 14:49:04.929972 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:04.929931    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:04.930143 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:04.930094    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 14:49:04.930182 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:04.930176    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert podName:f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce nodeName:}" failed. No retries permitted until 2026-07-30 14:49:08.9301549 +0000 UTC m=+54.603600998 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-jbpq8" (UID: "f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce") : secret "networking-console-plugin-cert" not found
Jul 30 14:49:05.001596 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:05.001553    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx" event={"ID":"bea29110-940e-491a-844b-966b31f8dea2","Type":"ContainerStarted","Data":"70ba77759259806d3ae08756745d3d4ca9bedbea68900759ba4002c5239d979d"}
Jul 30 14:49:05.002997 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:05.002966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql" event={"ID":"2b9a1df1-f0ea-4efb-b71d-ab8a20761180","Type":"ContainerStarted","Data":"3921206e4f38f40e8e861b17b6a4e688c73a6b2922ab93565eef25d248aef9fb"}
Jul 30 14:49:05.018599 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:05.018556    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-8z9mx" podStartSLOduration=1.2676668709999999 podStartE2EDuration="4.018544285s" totalImagesPullingTime="2.750877414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:05.018023143 +0000 UTC m=+50.691469275" watchObservedRunningTime="2026-07-30 14:49:05.018544285 +0000 UTC m=+50.691990398"
Jul 30 14:49:05.032694 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:05.032643    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql" podStartSLOduration=1.155607667 podStartE2EDuration="4.032624998s" totalImagesPullingTime="2.877017331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:05.032087289 +0000 UTC m=+50.705533420" watchObservedRunningTime="2026-07-30 14:49:05.032624998 +0000 UTC m=+50.706071112"
Jul 30 14:49:05.106701 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:05.106610    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:05.208516 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:05.208478    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:06.083712 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.083664    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:06.185209 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.185177    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:06.193063 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.193041    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2dlwr"]
Jul 30 14:49:06.214054 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.214021    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2dlwr"]
Jul 30 14:49:06.214193 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.214120    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.215983 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.215964    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9mfqr\""
Jul 30 14:49:06.216240 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.216214    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 14:49:06.216640 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.216617    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 14:49:06.216714 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.216652    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 14:49:06.216714 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.216683    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 14:49:06.260931 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.260904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:06.342842 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.342768    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.342842 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.342822    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/18005be7-0022-4919-81ff-9cc06fc5aa5b-data-volume\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.343012 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.342849    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.343012 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.342887    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn5s6\" (UniqueName: \"kubernetes.io/projected/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-api-access-nn5s6\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.343012 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.342952    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/18005be7-0022-4919-81ff-9cc06fc5aa5b-crio-socket\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.443857 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.443817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.443857 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.443857    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/18005be7-0022-4919-81ff-9cc06fc5aa5b-data-volume\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.444055 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.443875    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.444055 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.443918    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn5s6\" (UniqueName: \"kubernetes.io/projected/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-api-access-nn5s6\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.444055 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:06.443967    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:06.444055 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.443985    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/18005be7-0022-4919-81ff-9cc06fc5aa5b-crio-socket\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.444055 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:06.444035    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:06.944018834 +0000 UTC m=+52.617464925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:06.444055 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:06.444043    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 14:49:06.444264 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:06.444105    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:06.944088414 +0000 UTC m=+52.617534504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : configmap "kube-rbac-proxy" not found
Jul 30 14:49:06.444264 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.444184    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/18005be7-0022-4919-81ff-9cc06fc5aa5b-crio-socket\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.452170 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.452148    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn5s6\" (UniqueName: \"kubernetes.io/projected/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-api-access-nn5s6\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.455830 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.455808    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/18005be7-0022-4919-81ff-9cc06fc5aa5b-data-volume\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.947306 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.947269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.947306 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:06.947308    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:06.947548 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:06.947417    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 14:49:06.947548 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:06.947439    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:06.947548 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:06.947475    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:07.947459912 +0000 UTC m=+53.620906003 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : configmap "kube-rbac-proxy" not found
Jul 30 14:49:06.947548 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:06.947506    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:07.947500507 +0000 UTC m=+53.620946598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:07.955110 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:07.955070    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:07.955566 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:07.955207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:07.955566 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:07.955221    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 14:49:07.955566 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:07.955287    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:09.955271149 +0000 UTC m=+55.628717239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : configmap "kube-rbac-proxy" not found
Jul 30 14:49:07.955566 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:07.955302    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:07.955566 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:07.955350    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:09.955332411 +0000 UTC m=+55.628778523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:08.266464 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:08.266355    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:08.532201 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:08.532118    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:08.640222 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:08.640190    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:08.862388 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:08.862306    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:08.862388 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:08.862341    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:08.862572 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:08.862451    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 14:49:08.862572 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:08.862483    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:16.862465666 +0000 UTC m=+62.535911760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : configmap references non-existent config key: service-ca.crt
Jul 30 14:49:08.862572 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:08.862504    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:16.862497115 +0000 UTC m=+62.535943206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : secret "router-metrics-certs-default" not found
Jul 30 14:49:08.956308 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:08.956280    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:08.963067 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:08.963047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:08.963200 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:08.963184    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 14:49:08.963288 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:08.963249    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert podName:f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce nodeName:}" failed. No retries permitted until 2026-07-30 14:49:16.963231067 +0000 UTC m=+62.636677160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-jbpq8" (UID: "f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce") : secret "networking-console-plugin-cert" not found
Jul 30 14:49:09.071441 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:09.071382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:09.189284 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:09.189258    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vhmks_19783e54-8469-4c0d-906d-f30863b87ea6/dns-node-resolver/0.log"
Jul 30 14:49:09.971336 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:09.971290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:09.971336 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:09.971340    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:09.971828 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:09.971480    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:09.971828 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:09.971488    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 14:49:09.971828 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:09.971553    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:13.971536534 +0000 UTC m=+59.644982644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : configmap "kube-rbac-proxy" not found
Jul 30 14:49:09.971828 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:09.971567    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:13.97156163 +0000 UTC m=+59.645007721 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:10.194689 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:10.194663    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x2w25_64206ee8-2cf7-42dd-9771-4ee52fd66a97/node-ca/0.log"
Jul 30 14:49:10.269695 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:10.269599    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:10.329490 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:10.329462    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:10.603408 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:10.603309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:10.717679 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:10.717649    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:10.989826 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:10.989802    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rzcsk_cb36fd80-2c52-40fd-b9bf-4868a44b4030/serve-healthcheck-canary/0.log"
Jul 30 14:49:11.489605 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:11.489572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:11.590001 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:11.589962    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql_2b9a1df1-f0ea-4efb-b71d-ab8a20761180/kube-storage-version-migrator-operator/0.log"
Jul 30 14:49:14.001038 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:14.000997    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:14.001038 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:14.001041    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:14.001675 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:14.001112    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 14:49:14.001675 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:14.001136    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:14.001675 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:14.001186    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:22.001164241 +0000 UTC m=+67.674610353 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : configmap "kube-rbac-proxy" not found
Jul 30 14:49:14.001675 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:14.001206    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:22.001196195 +0000 UTC m=+67.674642286 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:14.373202 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:14.373122    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:14.510171 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:14.510136    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:14.610946 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:14.610916    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:14.718209 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:14.718181    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:14.815525 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:14.815492    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:14.947877 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:14.947847    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z46dq"
Jul 30 14:49:15.390222 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:15.390191    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:15.543272 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:15.543243    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:15.873606 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:15.873574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:16.502560 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:16.502530    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:16.674878 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:16.674847    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:16.866733 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:16.866648    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:16.922611 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:16.922577    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:16.922777 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:16.922616    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:16.922777 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:16.922749    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:32.922731848 +0000 UTC m=+78.596177944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : configmap references non-existent config key: service-ca.crt
Jul 30 14:49:16.922777 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:16.922755    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 14:49:16.922910 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:16.922809    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs podName:f31dcb58-4fd7-425b-b5a8-51dfac54437f nodeName:}" failed. No retries permitted until 2026-07-30 14:49:32.922794256 +0000 UTC m=+78.596240368 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs") pod "router-default-76489d7779-qr8pc" (UID: "f31dcb58-4fd7-425b-b5a8-51dfac54437f") : secret "router-metrics-certs-default" not found
Jul 30 14:49:17.023339 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:17.023299    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:17.023474 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:17.023458    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 14:49:17.023531 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:17.023522    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert podName:f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce nodeName:}" failed. No retries permitted until 2026-07-30 14:49:33.023506573 +0000 UTC m=+78.696952663 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-jbpq8" (UID: "f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce") : secret "networking-console-plugin-cert" not found
Jul 30 14:49:17.273502 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:17.273464    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:17.878125 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:17.878094    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:18.640060 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:18.640028    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:18.739642 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:18.739613    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:18.870740 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:18.870713    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:19.076925 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:19.076895    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:19.177544 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:19.177510    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:19.301047 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:19.301014    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:20.125166 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.125137    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:20.452615 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.452586    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:49:20.454762 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.454746    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 14:49:20.465429 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.465388    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/478b6274-9630-42e8-b363-e3e088f1bdbb-original-pull-secret\") pod \"global-pull-secret-syncer-n8tbb\" (UID: \"478b6274-9630-42e8-b363-e3e088f1bdbb\") " pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:49:20.474946 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.474919    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:20.553342 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.553310    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7z2g\" (UniqueName: \"kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g\") pod \"network-check-target-9tjwk\" (UID: \"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85\") " pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:49:20.553484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.553387    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:49:20.553549 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.553508    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:49:20.555157 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.555136    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 14:49:20.555415 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.555383    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 14:49:20.555840 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.555825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7cd0a49-d9f2-42d7-9f3a-abc723a0d944-metrics-tls\") pod \"dns-default-jjcpx\" (UID: \"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944\") " pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:49:20.563958 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:20.563935    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 30 14:49:20.564036 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:20.563999    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs podName:4be90577-11f7-4070-878c-7a6f277a401b nodeName:}" failed. No retries permitted until 2026-07-30 14:50:24.563979772 +0000 UTC m=+130.237425883 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs") pod "network-metrics-daemon-9vx62" (UID: "4be90577-11f7-4070-878c-7a6f277a401b") : secret "metrics-daemon-secret" not found
Jul 30 14:49:20.565597 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.565581    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 14:49:20.576694 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.576674    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7z2g\" (UniqueName: \"kubernetes.io/projected/2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85-kube-api-access-f7z2g\") pod \"network-check-target-9tjwk\" (UID: \"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85\") " pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:49:20.619826 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.619800    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8tbb"
Jul 30 14:49:20.627715 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.627697    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dd6sm\""
Jul 30 14:49:20.636283 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.636263    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:49:20.680694 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.680660    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:20.747507 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.747434    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n8tbb"]
Jul 30 14:49:20.751182 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:20.751156    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478b6274_9630_42e8_b363_e3e088f1bdbb.slice/crio-efedb0a64ef26e44d0a0d21cab63a3597612e702a5e33026694670be5de5a69f WatchSource:0}: Error finding container efedb0a64ef26e44d0a0d21cab63a3597612e702a5e33026694670be5de5a69f: Status 404 returned error can't find the container with id efedb0a64ef26e44d0a0d21cab63a3597612e702a5e33026694670be5de5a69f
Jul 30 14:49:20.766230 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.766206    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9tjwk"]
Jul 30 14:49:20.769139 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:20.769107    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b5c5d13_cc42_4795_a0aa_3f90ac0d3c85.slice/crio-50062decbcbc6dec4e79337a17f9694c98822ed38267a76302d94d8224017ff6 WatchSource:0}: Error finding container 50062decbcbc6dec4e79337a17f9694c98822ed38267a76302d94d8224017ff6: Status 404 returned error can't find the container with id 50062decbcbc6dec4e79337a17f9694c98822ed38267a76302d94d8224017ff6
Jul 30 14:49:20.811421 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.811388    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vrbw2\""
Jul 30 14:49:20.820115 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.820098    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:49:20.871503 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.871475    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:20.944197 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:20.944164    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jjcpx"]
Jul 30 14:49:20.947905 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:20.947880    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7cd0a49_d9f2_42d7_9f3a_abc723a0d944.slice/crio-9d96e2ea8574869d8e26788d7e94bb0194b26c7093860d8aaf301949476bb6e2 WatchSource:0}: Error finding container 9d96e2ea8574869d8e26788d7e94bb0194b26c7093860d8aaf301949476bb6e2: Status 404 returned error can't find the container with id 9d96e2ea8574869d8e26788d7e94bb0194b26c7093860d8aaf301949476bb6e2
Jul 30 14:49:21.011357 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:21.011285    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:21.040061 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:21.040012    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjcpx" event={"ID":"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944","Type":"ContainerStarted","Data":"9d96e2ea8574869d8e26788d7e94bb0194b26c7093860d8aaf301949476bb6e2"}
Jul 30 14:49:21.041088 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:21.041064    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n8tbb" event={"ID":"478b6274-9630-42e8-b363-e3e088f1bdbb","Type":"ContainerStarted","Data":"efedb0a64ef26e44d0a0d21cab63a3597612e702a5e33026694670be5de5a69f"}
Jul 30 14:49:21.042112 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:21.042094    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9tjwk" event={"ID":"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85","Type":"ContainerStarted","Data":"50062decbcbc6dec4e79337a17f9694c98822ed38267a76302d94d8224017ff6"}
Jul 30 14:49:21.112183 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:21.112153    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:21.368930 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:21.368813    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:21.956693 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:21.956657    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:22.067843 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:22.067803    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:22.067843 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:22.067856    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:22.068118 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:22.067966    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:22.068118 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:49:22.068073    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls podName:18005be7-0022-4919-81ff-9cc06fc5aa5b nodeName:}" failed. No retries permitted until 2026-07-30 14:49:38.068051888 +0000 UTC m=+83.741497984 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2dlwr" (UID: "18005be7-0022-4919-81ff-9cc06fc5aa5b") : secret "insights-runtime-extractor-tls" not found
Jul 30 14:49:22.068596 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:22.068559    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/18005be7-0022-4919-81ff-9cc06fc5aa5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:22.277612 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:22.277493    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:22.674630 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:22.674515    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:22.869386 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:22.869354    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:23.274177 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:23.274151    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:23.340591 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:23.340562    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:23.904108 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:23.904067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:24.065542 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:24.065500    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:24.126264 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:24.126219    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:24.222689 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:24.222661    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:24.322898 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:24.322869    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:24.423503 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:24.423466    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:24.584711 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:24.584614    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:24.685919 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:24.685886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:24.751648 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:24.751600    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:24.871706 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:24.871610    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:25.074509 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:25.074472    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:25.607452 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:25.607419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:25.788818 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:25.788763    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:25.957345 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:25.957287    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:26.058023 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:26.057992    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:26.059462 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:26.059434    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjcpx" event={"ID":"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944","Type":"ContainerStarted","Data":"6f9c9c5491fb8e4bf4ae9bcc4b048d6dfae66a6b01e95061f21e93331652876e"}
Jul 30 14:49:26.059579 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:26.059523    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjcpx" event={"ID":"e7cd0a49-d9f2-42d7-9f3a-abc723a0d944","Type":"ContainerStarted","Data":"798a67054835059fca5a2cd6a27ae0f19fe081cf0c386f3f34b91bcc6a5eab46"}
Jul 30 14:49:26.059579 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:26.059556    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:49:26.060852 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:26.060824    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n8tbb" event={"ID":"478b6274-9630-42e8-b363-e3e088f1bdbb","Type":"ContainerStarted","Data":"31f9d1515e50e9ed595a1c26e901166cac997b97cb1e70c6ac20cfd45dd7000d"}
Jul 30 14:49:26.062311 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:26.062289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9tjwk" event={"ID":"2b5c5d13-cc42-4795-a0aa-3f90ac0d3c85","Type":"ContainerStarted","Data":"707330163e3fa1b1f80e646fda75bd2342ed6496a60f5edfe5f27fffb2534dcb"}
Jul 30 14:49:26.062445 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:26.062419    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:49:26.077858 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:26.077809    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jjcpx" podStartSLOduration=33.272267098 podStartE2EDuration="38.077794873s" totalImagesPullingTime="4.805527775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:26.076829892 +0000 UTC m=+71.750276002" watchObservedRunningTime="2026-07-30 14:49:26.077794873 +0000 UTC m=+71.751240963"
Jul 30 14:49:26.092481 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:26.092440    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9tjwk" podStartSLOduration=67.101464173 podStartE2EDuration="1m12.092426425s" totalImagesPullingTime="4.990962252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:26.091770138 +0000 UTC m=+71.765216253" watchObservedRunningTime="2026-07-30 14:49:26.092426425 +0000 UTC m=+71.765872537"
Jul 30 14:49:26.105944 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:26.105904    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n8tbb" podStartSLOduration=67.089081529 podStartE2EDuration="1m12.105890284s" totalImagesPullingTime="5.016808755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:26.105709696 +0000 UTC m=+71.779155812" watchObservedRunningTime="2026-07-30 14:49:26.105890284 +0000 UTC m=+71.779336399"
Jul 30 14:49:29.809265 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.809226    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"]
Jul 30 14:49:29.812428 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.812389    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"]
Jul 30 14:49:29.812554 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.812506    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"
Jul 30 14:49:29.814486 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.814464    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 30 14:49:29.814486 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.814474    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xrlsp\""
Jul 30 14:49:29.814689 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.814497    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 14:49:29.814689 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.814557    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 14:49:29.814809 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.814793    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 14:49:29.815523 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.815506    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:29.817249 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.817229    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 30 14:49:29.817353 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.817234    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 30 14:49:29.817531 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.817517    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 30 14:49:29.817608 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.817576    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 30 14:49:29.823135 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.823119    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"]
Jul 30 14:49:29.826446 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.826426    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"]
Jul 30 14:49:29.931037 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.931003    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:29.931223 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.931081    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:29.931223 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.931122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/696edcb9-d603-43a4-becf-06005cd53e1b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-668747d964-2wmn5\" (UID: \"696edcb9-d603-43a4-becf-06005cd53e1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"
Jul 30 14:49:29.931223 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.931159    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-ca\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:29.931223 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.931197    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nb9k\" (UniqueName: \"kubernetes.io/projected/696edcb9-d603-43a4-becf-06005cd53e1b-kube-api-access-4nb9k\") pod \"managed-serviceaccount-addon-agent-668747d964-2wmn5\" (UID: \"696edcb9-d603-43a4-becf-06005cd53e1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"
Jul 30 14:49:29.931385 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.931265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/701610e0-1b6d-4578-b826-238920ee49ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:29.931385 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.931303    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdqsd\" (UniqueName: \"kubernetes.io/projected/701610e0-1b6d-4578-b826-238920ee49ef-kube-api-access-rdqsd\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:29.931385 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.931330    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-hub\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:29.955486 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:29.955452    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:30.032235 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.032207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-hub\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.032362 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.032239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.032362 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.032274    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.032554 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.032420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/696edcb9-d603-43a4-becf-06005cd53e1b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-668747d964-2wmn5\" (UID: \"696edcb9-d603-43a4-becf-06005cd53e1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"
Jul 30 14:49:30.032554 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.032475    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-ca\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.032554 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.032511    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nb9k\" (UniqueName: \"kubernetes.io/projected/696edcb9-d603-43a4-becf-06005cd53e1b-kube-api-access-4nb9k\") pod \"managed-serviceaccount-addon-agent-668747d964-2wmn5\" (UID: \"696edcb9-d603-43a4-becf-06005cd53e1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"
Jul 30 14:49:30.032554 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.032551    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/701610e0-1b6d-4578-b826-238920ee49ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.033324 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.032848    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdqsd\" (UniqueName: \"kubernetes.io/projected/701610e0-1b6d-4578-b826-238920ee49ef-kube-api-access-rdqsd\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.033481 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.033459    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/701610e0-1b6d-4578-b826-238920ee49ef-additional-service-ca\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.035046 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.035023    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-hub\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.035122 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.035066    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-ca\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.035177 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.035159    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.035419 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.035380    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/701610e0-1b6d-4578-b826-238920ee49ef-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.035585 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.035571    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/696edcb9-d603-43a4-becf-06005cd53e1b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-668747d964-2wmn5\" (UID: \"696edcb9-d603-43a4-becf-06005cd53e1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"
Jul 30 14:49:30.040023 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.039996    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nb9k\" (UniqueName: \"kubernetes.io/projected/696edcb9-d603-43a4-becf-06005cd53e1b-kube-api-access-4nb9k\") pod \"managed-serviceaccount-addon-agent-668747d964-2wmn5\" (UID: \"696edcb9-d603-43a4-becf-06005cd53e1b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"
Jul 30 14:49:30.040268 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.040253    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdqsd\" (UniqueName: \"kubernetes.io/projected/701610e0-1b6d-4578-b826-238920ee49ef-kube-api-access-rdqsd\") pod \"cluster-proxy-proxy-agent-554f99b7df-mhp4r\" (UID: \"701610e0-1b6d-4578-b826-238920ee49ef\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.052482 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.052462    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:30.128144 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.128094    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"
Jul 30 14:49:30.132919 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.132904    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"
Jul 30 14:49:30.193569 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.193387    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:30.266739 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.266710    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5"]
Jul 30 14:49:30.269695 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:30.269671    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod696edcb9_d603_43a4_becf_06005cd53e1b.slice/crio-fb7c86cb2e8ee138702d343f061422ca8bdc93d77573fda90c3344412d9427ac WatchSource:0}: Error finding container fb7c86cb2e8ee138702d343f061422ca8bdc93d77573fda90c3344412d9427ac: Status 404 returned error can't find the container with id fb7c86cb2e8ee138702d343f061422ca8bdc93d77573fda90c3344412d9427ac
Jul 30 14:49:30.283507 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:30.283485    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r"]
Jul 30 14:49:30.286762 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:30.286739    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod701610e0_1b6d_4578_b826_238920ee49ef.slice/crio-5df25af6ec77de6d6ab207c63df894a08e912a3e8d3cb9c396e6b8dda1a429f3 WatchSource:0}: Error finding container 5df25af6ec77de6d6ab207c63df894a08e912a3e8d3cb9c396e6b8dda1a429f3: Status 404 returned error can't find the container with id 5df25af6ec77de6d6ab207c63df894a08e912a3e8d3cb9c396e6b8dda1a429f3
Jul 30 14:49:31.081589 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:31.081542    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r" event={"ID":"701610e0-1b6d-4578-b826-238920ee49ef","Type":"ContainerStarted","Data":"5df25af6ec77de6d6ab207c63df894a08e912a3e8d3cb9c396e6b8dda1a429f3"}
Jul 30 14:49:31.083155 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:31.083122    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5" event={"ID":"696edcb9-d603-43a4-becf-06005cd53e1b","Type":"ContainerStarted","Data":"fb7c86cb2e8ee138702d343f061422ca8bdc93d77573fda90c3344412d9427ac"}
Jul 30 14:49:32.959592 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:32.959553    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:32.960012 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:32.959652    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:32.960623 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:32.960271    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f31dcb58-4fd7-425b-b5a8-51dfac54437f-service-ca-bundle\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:32.963248 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:32.963226    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f31dcb58-4fd7-425b-b5a8-51dfac54437f-metrics-certs\") pod \"router-default-76489d7779-qr8pc\" (UID: \"f31dcb58-4fd7-425b-b5a8-51dfac54437f\") " pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:33.060693 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:33.060657    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:33.063364 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:33.063342    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-jbpq8\" (UID: \"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:33.221708 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:33.221638    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jpgsk\""
Jul 30 14:49:33.230324 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:33.230294    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:33.283949 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:33.283919    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hdgfm\""
Jul 30 14:49:33.292234 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:33.292210    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"
Jul 30 14:49:33.624328 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:33.624249    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:33.688726 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:33.688689    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:35.104882 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:35.103173    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r" event={"ID":"701610e0-1b6d-4578-b826-238920ee49ef","Type":"ContainerStarted","Data":"5966d1d81d02b193aa0d0340072b840a6d83a7e2a3c6bb18a2c313a738e136b5"}
Jul 30 14:49:35.105946 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:35.105902    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5" event={"ID":"696edcb9-d603-43a4-becf-06005cd53e1b","Type":"ContainerStarted","Data":"7e75d661ed03362ad1f579343e8199cbc5bdbce6e845ae174cd14c5a7acc6b17"}
Jul 30 14:49:35.121460 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:35.121410    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-668747d964-2wmn5" podStartSLOduration=1.409832903 podStartE2EDuration="6.121367997s" totalImagesPullingTime="4.711535094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:35.121157496 +0000 UTC m=+80.794603628" watchObservedRunningTime="2026-07-30 14:49:35.121367997 +0000 UTC m=+80.794814111"
Jul 30 14:49:35.127669 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:35.127533    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8"]
Jul 30 14:49:35.132043 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:35.131988    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95cd5c5_f2a4_4e2d_bd6b_5e75bea9abce.slice/crio-3d06fda1865854a0a183fc34c744f6c9c2f94a78f35a7882a53157d8f5014e82 WatchSource:0}: Error finding container 3d06fda1865854a0a183fc34c744f6c9c2f94a78f35a7882a53157d8f5014e82: Status 404 returned error can't find the container with id 3d06fda1865854a0a183fc34c744f6c9c2f94a78f35a7882a53157d8f5014e82
Jul 30 14:49:35.154597 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:35.154563    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76489d7779-qr8pc"]
Jul 30 14:49:35.159674 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:35.159634    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf31dcb58_4fd7_425b_b5a8_51dfac54437f.slice/crio-10fd391b92050250343a6c9c288ea22739f8a63a752cc5e3876896ef6ad32157 WatchSource:0}: Error finding container 10fd391b92050250343a6c9c288ea22739f8a63a752cc5e3876896ef6ad32157: Status 404 returned error can't find the container with id 10fd391b92050250343a6c9c288ea22739f8a63a752cc5e3876896ef6ad32157
Jul 30 14:49:36.068524 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.068378    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jjcpx"
Jul 30 14:49:36.112127 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.112085    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76489d7779-qr8pc" event={"ID":"f31dcb58-4fd7-425b-b5a8-51dfac54437f","Type":"ContainerStarted","Data":"cdc003ff99e5a024bb602c1c05b111622f59e4e773bc3b4ad8790333456a5e5d"}
Jul 30 14:49:36.112518 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.112153    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76489d7779-qr8pc" event={"ID":"f31dcb58-4fd7-425b-b5a8-51dfac54437f","Type":"ContainerStarted","Data":"10fd391b92050250343a6c9c288ea22739f8a63a752cc5e3876896ef6ad32157"}
Jul 30 14:49:36.113455 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.113423    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8" event={"ID":"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce","Type":"ContainerStarted","Data":"3d06fda1865854a0a183fc34c744f6c9c2f94a78f35a7882a53157d8f5014e82"}
Jul 30 14:49:36.115902 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.115874    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r" event={"ID":"701610e0-1b6d-4578-b826-238920ee49ef","Type":"ContainerStarted","Data":"4b5aca1bea92e6345b15bc2c02379e2ef6cc79eda62655939a4a3f19125f1ed5"}
Jul 30 14:49:36.116023 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.115918    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r" event={"ID":"701610e0-1b6d-4578-b826-238920ee49ef","Type":"ContainerStarted","Data":"a508da53b05d16e8a5e2a4b2abba6cc9c945b912e51412f7843be21828e2689a"}
Jul 30 14:49:36.150932 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.150867    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-76489d7779-qr8pc" podStartSLOduration=35.150846357 podStartE2EDuration="35.150846357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:36.130661285 +0000 UTC m=+81.804107428" watchObservedRunningTime="2026-07-30 14:49:36.150846357 +0000 UTC m=+81.824292472"
Jul 30 14:49:36.151486 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.151447    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554f99b7df-mhp4r" podStartSLOduration=2.4408257620000002 podStartE2EDuration="7.151439745s" totalImagesPullingTime="4.710613983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:36.149846507 +0000 UTC m=+81.823292620" watchObservedRunningTime="2026-07-30 14:49:36.151439745 +0000 UTC m=+81.824885859"
Jul 30 14:49:36.231035 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.230997    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:36.233486 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.233463    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:36.502436 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.502407    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:36.602439 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.602387    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:36.672109 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.672076    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:36.895303 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:36.895270    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:37.120706 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:37.120669    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8" event={"ID":"f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce","Type":"ContainerStarted","Data":"b02192a436c2432e32bb6713333941fdd21106ed6afbaa3889fdbc7246730a09"}
Jul 30 14:49:37.121122 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:37.121047    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:37.122275 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:37.122255    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-76489d7779-qr8pc"
Jul 30 14:49:37.180385 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:37.180281    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-jbpq8" podStartSLOduration=34.890219065 podStartE2EDuration="36.180259288s" totalImagesPullingTime="1.290040223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:37.147476829 +0000 UTC m=+82.820922942" watchObservedRunningTime="2026-07-30 14:49:37.180259288 +0000 UTC m=+82.853705407"
Jul 30 14:49:37.353629 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:37.353586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:37.451169 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:37.451142    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:37.513181 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:37.513150    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:38.113119 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:38.113079    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:38.115465 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:38.115446    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/18005be7-0022-4919-81ff-9cc06fc5aa5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2dlwr\" (UID: \"18005be7-0022-4919-81ff-9cc06fc5aa5b\") " pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:38.325733 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:38.325703    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9mfqr\""
Jul 30 14:49:38.333357 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:38.333338    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2dlwr"
Jul 30 14:49:38.478048 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:38.478019    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2dlwr"]
Jul 30 14:49:38.481235 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:38.481211    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18005be7_0022_4919_81ff_9cc06fc5aa5b.slice/crio-ccbfdb9df6fbe3fba65510ab4f1cd14242228b1743407012dca26c91e3dae63a WatchSource:0}: Error finding container ccbfdb9df6fbe3fba65510ab4f1cd14242228b1743407012dca26c91e3dae63a: Status 404 returned error can't find the container with id ccbfdb9df6fbe3fba65510ab4f1cd14242228b1743407012dca26c91e3dae63a
Jul 30 14:49:39.130252 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:39.130215    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2dlwr" event={"ID":"18005be7-0022-4919-81ff-9cc06fc5aa5b","Type":"ContainerStarted","Data":"ccbfdb9df6fbe3fba65510ab4f1cd14242228b1743407012dca26c91e3dae63a"}
Jul 30 14:49:40.695248 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:40.695208    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:40.769248 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:40.769215    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:41.216537 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:41.216509    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:43.266194 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:43.266164    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:43.349694 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:43.349666    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:43.825922 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:43.825888    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:43.925561 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:43.925526    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:44.151711 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:44.151629    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2dlwr" event={"ID":"18005be7-0022-4919-81ff-9cc06fc5aa5b","Type":"ContainerStarted","Data":"b8ab1053e687600ca27096fc168d1093569e4411f2be3675043ce9ce8ea495a9"}
Jul 30 14:49:44.821825 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:44.821783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:45.198450 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:45.198418    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:45.272131 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:45.272089    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:46.678503 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:46.678456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:46.779486 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:46.779458    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:46.878120 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:46.878092    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:47.164008 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:47.163979    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2dlwr" event={"ID":"18005be7-0022-4919-81ff-9cc06fc5aa5b","Type":"ContainerStarted","Data":"24f364adfaa595a7508a04ec9dabcaf33fdaa81af8e4bd107f7d2502ec6ff13d"}
Jul 30 14:49:47.612548 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:47.612518    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:47.713058 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:47.713027    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:48.910508 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:48.910481    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:50.821386 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:50.821357    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:50.948032 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:50.948000    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:51.109688 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.109651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:51.137240 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.137207    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ccbwd"]
Jul 30 14:49:51.140474 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.140453    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.142914 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.142893    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 14:49:51.143492 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.143471    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m4t2j\""
Jul 30 14:49:51.143597 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.143481    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 14:49:51.143597 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.143571    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 14:49:51.143701 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.143523    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 14:49:51.143754 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.143704    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 14:49:51.144039 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.144020    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 14:49:51.180447 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.180416    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2dlwr" event={"ID":"18005be7-0022-4919-81ff-9cc06fc5aa5b","Type":"ContainerStarted","Data":"c7f6918bd4466aea3c4410f9d900d1b78bc68be04a69dda2a6c7796b3875cb3f"}
Jul 30 14:49:51.202668 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.202579    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2dlwr" podStartSLOduration=33.181323043 podStartE2EDuration="45.202562902s" totalImagesPullingTime="12.021239859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 14:49:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:51.200809286 +0000 UTC m=+96.874255409" watchObservedRunningTime="2026-07-30 14:49:51.202562902 +0000 UTC m=+96.876009016"
Jul 30 14:49:51.213524 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.213499    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:51.279508 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.279481    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:51.313570 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.313546    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-sys\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.313660 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.313596    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.313660 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.313628    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-wtmp\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.313660 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.313651    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-metrics-client-ca\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.313753 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.313696    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.313753 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.313748    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-textfile\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.313856 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.313832    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdxz7\" (UniqueName: \"kubernetes.io/projected/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-kube-api-access-qdxz7\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.313891 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.313872    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-root\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.313924 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.313913    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-tls\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.381488 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.381465    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:51.415246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415214    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415378 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-wtmp\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415378 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-metrics-client-ca\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415378 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415305    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415378 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415351    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-textfile\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415607 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415386    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdxz7\" (UniqueName: \"kubernetes.io/projected/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-kube-api-access-qdxz7\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415607 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415429    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-root\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415607 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415449    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-wtmp\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415607 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415470    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-tls\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415607 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415520    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-sys\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415607 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415541    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-root\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415889 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415615    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-sys\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.415976 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415956    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-metrics-client-ca\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.416033 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.415999    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-accelerators-collector-config\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.416259 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.416197    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-textfile\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.417918 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.417794    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.418190 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.418168    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-node-exporter-tls\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.426951 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.426922    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdxz7\" (UniqueName: \"kubernetes.io/projected/e53f9fde-be7d-43e1-98c2-48bf0148b3e4-kube-api-access-qdxz7\") pod \"node-exporter-ccbwd\" (UID: \"e53f9fde-be7d-43e1-98c2-48bf0148b3e4\") " pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.450111 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.450087    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ccbwd"
Jul 30 14:49:51.461590 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:49:51.461563    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53f9fde_be7d_43e1_98c2_48bf0148b3e4.slice/crio-a877db3ceb75e02f6fbed9e83ebe06ec13312d42ab7c7b1d601dfeef437236c8 WatchSource:0}: Error finding container a877db3ceb75e02f6fbed9e83ebe06ec13312d42ab7c7b1d601dfeef437236c8: Status 404 returned error can't find the container with id a877db3ceb75e02f6fbed9e83ebe06ec13312d42ab7c7b1d601dfeef437236c8
Jul 30 14:49:51.480284 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.480258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:51.579442 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.579416    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:51.680244 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:51.680210    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:52.062674 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:52.062637    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:52.157113 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:52.157075    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:52.185435 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:52.185385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ccbwd" event={"ID":"e53f9fde-be7d-43e1-98c2-48bf0148b3e4","Type":"ContainerStarted","Data":"a877db3ceb75e02f6fbed9e83ebe06ec13312d42ab7c7b1d601dfeef437236c8"}
Jul 30 14:49:52.390547 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:52.390466    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:52.469113 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:52.469081    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:52.561812 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:52.561791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:52.662229 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:52.662207    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:52.729277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:52.729255    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:53.058576 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:53.058548    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:53.145723 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:53.145691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:53.190271 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:53.190240    2582 generic.go:356] "Generic (PLEG): container finished" podID="e53f9fde-be7d-43e1-98c2-48bf0148b3e4" containerID="6c4959c379a66c9e78eba1a463664ffe40811192eae40ff951a7cf7a1b336840" exitCode=0
Jul 30 14:49:53.190451 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:53.190328    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ccbwd" event={"ID":"e53f9fde-be7d-43e1-98c2-48bf0148b3e4","Type":"ContainerDied","Data":"6c4959c379a66c9e78eba1a463664ffe40811192eae40ff951a7cf7a1b336840"}
Jul 30 14:49:53.434193 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:53.434163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:53.542089 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:53.542008    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:54.169694 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:54.169657    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:54.195807 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:54.195770    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ccbwd" event={"ID":"e53f9fde-be7d-43e1-98c2-48bf0148b3e4","Type":"ContainerStarted","Data":"bf2a82825a56c9863c807eda66b476c65e5f3ba38026c48eb3ee6ac52428dd03"}
Jul 30 14:49:54.195937 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:54.195852    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ccbwd" event={"ID":"e53f9fde-be7d-43e1-98c2-48bf0148b3e4","Type":"ContainerStarted","Data":"3dc5aecf4e92812bb6174876a907381d716b71e1e66b269e5666180b3af51936"}
Jul 30 14:49:54.237683 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:54.237640    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ccbwd" podStartSLOduration=2.120020861 podStartE2EDuration="3.237627235s" totalImagesPullingTime="1.047149422s" totalInitContainerRuntime="70.456952ms" isStatefulPod=true podCreationTimestamp="2026-07-30 14:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:49:54.23702404 +0000 UTC m=+99.910470152" watchObservedRunningTime="2026-07-30 14:49:54.237627235 +0000 UTC m=+99.911073347"
Jul 30 14:49:54.270779 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:54.270757    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:54.374702 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:54.374651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:54.571475 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:54.571440    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:55.370969 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:55.370942    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:55.470045 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:55.470011    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:55.544060 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:55.544031    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:55.881364 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:55.881328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:55.979791 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:55.979760    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:56.518171 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:56.518141    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:57.068863 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:57.068836    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9tjwk"
Jul 30 14:49:57.473970 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:57.473941    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:57.681380 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:57.681347    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:49:57.794543 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:49:57.794459    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:04.278848 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:04.278811    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:04.379390 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:04.379355    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:06.001308 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:06.001277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:06.101839 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:06.101806    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:07.136542 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:07.136509    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:07.237150 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:07.237118    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:11.252158 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:11.252122    2582 generic.go:356] "Generic (PLEG): container finished" podID="2b9a1df1-f0ea-4efb-b71d-ab8a20761180" containerID="3921206e4f38f40e8e861b17b6a4e688c73a6b2922ab93565eef25d248aef9fb" exitCode=0
Jul 30 14:50:11.252658 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:11.252201    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql" event={"ID":"2b9a1df1-f0ea-4efb-b71d-ab8a20761180","Type":"ContainerDied","Data":"3921206e4f38f40e8e861b17b6a4e688c73a6b2922ab93565eef25d248aef9fb"}
Jul 30 14:50:11.252729 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:11.252656    2582 scope.go:117] "RemoveContainer" containerID="3921206e4f38f40e8e861b17b6a4e688c73a6b2922ab93565eef25d248aef9fb"
Jul 30 14:50:12.257255 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:12.257224    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql" event={"ID":"2b9a1df1-f0ea-4efb-b71d-ab8a20761180","Type":"ContainerStarted","Data":"84380920e30029e8b67748241badab303518d316a524179c3adc6d8bcfd64d4b"}
Jul 30 14:50:12.321037 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:12.321000    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:12.384055 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:12.384020    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:12.441096 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:12.441065    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:18.709544 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:18.709514    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:18.773383 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:18.773353    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:21.022655 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:21.022620    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:21.098935 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:21.098905    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:21.735840 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:21.735805    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:21.835615 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:21.835585    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:24.564171 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:24.564131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:50:24.566669 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:24.566645    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4be90577-11f7-4070-878c-7a6f277a401b-metrics-certs\") pod \"network-metrics-daemon-9vx62\" (UID: \"4be90577-11f7-4070-878c-7a6f277a401b\") " pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:50:24.815484 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:24.815408    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-82hs8\""
Jul 30 14:50:24.824076 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:24.824058    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9vx62"
Jul 30 14:50:25.000365 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:25.000337    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9vx62"]
Jul 30 14:50:25.003819 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:50:25.003782    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4be90577_11f7_4070_878c_7a6f277a401b.slice/crio-4bc482008824ff965ba8be9f6f9483081fd097e4312fe2066ddae8001ddf5dd6 WatchSource:0}: Error finding container 4bc482008824ff965ba8be9f6f9483081fd097e4312fe2066ddae8001ddf5dd6: Status 404 returned error can't find the container with id 4bc482008824ff965ba8be9f6f9483081fd097e4312fe2066ddae8001ddf5dd6
Jul 30 14:50:25.300861 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:25.300832    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vx62" event={"ID":"4be90577-11f7-4070-878c-7a6f277a401b","Type":"ContainerStarted","Data":"4bc482008824ff965ba8be9f6f9483081fd097e4312fe2066ddae8001ddf5dd6"}
Jul 30 14:50:27.309433 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:27.309381    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vx62" event={"ID":"4be90577-11f7-4070-878c-7a6f277a401b","Type":"ContainerStarted","Data":"744d89e2f382548916969570d274d36fb0659877e288dbdab02e401bd0e8ee4d"}
Jul 30 14:50:27.309891 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:27.309452    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9vx62" event={"ID":"4be90577-11f7-4070-878c-7a6f277a401b","Type":"ContainerStarted","Data":"5b7180ccac5c4a00432b8e9dd19e96d822dd5f9422c4e323f1ae4e002d5326bf"}
Jul 30 14:50:27.327593 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:27.327532    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9vx62" podStartSLOduration=132.079213654 podStartE2EDuration="2m13.327507481s" totalImagesPullingTime="1.248293827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:50:27.325812167 +0000 UTC m=+132.999258469" watchObservedRunningTime="2026-07-30 14:50:27.327507481 +0000 UTC m=+133.000953595"
Jul 30 14:50:27.430792 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:27.430750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:27.488765 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:27.488734    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:36.015637 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:36.015598    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:36.277763 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:36.277645    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:50:40.514562 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:50:40.514524    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:01.496788 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:01.496751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:07.283554 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:07.283466    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:07.343049 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:07.343026    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:12.309500 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:12.309447    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:12.370648 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:12.370616    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:19.344629 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:19.344593    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:19.407496 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:19.407462    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:23.635637 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:23.635606    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:23.736976 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:23.736937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:23.797776 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:23.797750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:36.007815 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:36.007779    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:36.108994 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:36.108957    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:36.161728 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:36.161700    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:51:47.314105 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:51:47.314074    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:10.677191 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:10.677132    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:10.779653 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:10.779621    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:37.363341 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:37.363249    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:38.152636 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:38.152602    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:40.981558 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:40.981522    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:41.083902 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:41.083872    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:42.761786 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:42.761756    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:42.860356 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:42.860327    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:43.246296 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:43.246267    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:43.420675 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:43.420640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:45.164933 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:45.164900    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:45.571983 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:45.571954    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:48.375436 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:48.375382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:48.428093 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:48.428064    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:53.369651 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:53.369575    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:59.580852 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:59.580818    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:52:59.679376 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:52:59.679345    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:06.971180 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:06.971144    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:07.072856 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:07.072800    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:08.887587 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:08.887549    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:13.854071 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:13.854038    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:13.906647 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:13.906615    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:14.642016 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:14.641993    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 14:53:20.778364 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:20.778332    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:20.918161 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:20.918132    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:21.141926 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.141846    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:21.221529 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.221498    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"]
Jul 30 14:53:21.224429 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.224414    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.226340 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.226318    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:21.229432 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.229388    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 30 14:53:21.229432 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.229406    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 14:53:21.230414 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.230374    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 14:53:21.231252 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.231235    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 30 14:53:21.231909 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.231890    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 30 14:53:21.232430 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.232412    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-q26xj\""
Jul 30 14:53:21.253172 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.253150    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"]
Jul 30 14:53:21.374415 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.374366    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/add9f030-6c98-4832-a445-402a4926bdfb-metrics-cert\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.374590 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.374460    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjvp4\" (UniqueName: \"kubernetes.io/projected/add9f030-6c98-4832-a445-402a4926bdfb-kube-api-access-cjvp4\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.374590 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.374490    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/add9f030-6c98-4832-a445-402a4926bdfb-manager-config\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.374590 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.374515    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add9f030-6c98-4832-a445-402a4926bdfb-cert\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.432921 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.432891    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:21.475362 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.475338    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjvp4\" (UniqueName: \"kubernetes.io/projected/add9f030-6c98-4832-a445-402a4926bdfb-kube-api-access-cjvp4\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.475544 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.475375    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/add9f030-6c98-4832-a445-402a4926bdfb-manager-config\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.475544 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.475417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add9f030-6c98-4832-a445-402a4926bdfb-cert\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.475544 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.475462    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/add9f030-6c98-4832-a445-402a4926bdfb-metrics-cert\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.476047 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.476024    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/add9f030-6c98-4832-a445-402a4926bdfb-manager-config\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.478027 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.478009    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/add9f030-6c98-4832-a445-402a4926bdfb-metrics-cert\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.478119 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.478030    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/add9f030-6c98-4832-a445-402a4926bdfb-cert\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.484431 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.484385    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjvp4\" (UniqueName: \"kubernetes.io/projected/add9f030-6c98-4832-a445-402a4926bdfb-kube-api-access-cjvp4\") pod \"lws-controller-manager-7b696b7c-fj29g\" (UID: \"add9f030-6c98-4832-a445-402a4926bdfb\") " pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.533264 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.533229    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:21.661189 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.661156    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"]
Jul 30 14:53:21.664320 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:53:21.664295    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd9f030_6c98_4832_a445_402a4926bdfb.slice/crio-76223eb7c1ccb349feeedf4fe997a85fd8fb1382e2e7546b1bd70bfd3c60c585 WatchSource:0}: Error finding container 76223eb7c1ccb349feeedf4fe997a85fd8fb1382e2e7546b1bd70bfd3c60c585: Status 404 returned error can't find the container with id 76223eb7c1ccb349feeedf4fe997a85fd8fb1382e2e7546b1bd70bfd3c60c585
Jul 30 14:53:21.666146 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.666132    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 14:53:21.884982 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.884901    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g" event={"ID":"add9f030-6c98-4832-a445-402a4926bdfb","Type":"ContainerStarted","Data":"76223eb7c1ccb349feeedf4fe997a85fd8fb1382e2e7546b1bd70bfd3c60c585"}
Jul 30 14:53:21.968129 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:21.968104    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:22.072493 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.072462    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:22.118358 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.118326    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"]
Jul 30 14:53:22.122482 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.122464    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.126819 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.126792    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 14:53:22.126925 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.126887    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 14:53:22.126925 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.126886    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 14:53:22.127016 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.126968    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zsh4t\""
Jul 30 14:53:22.127133 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.127120    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 14:53:22.143872 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.143819    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"]
Jul 30 14:53:22.280980 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.280948    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/30327971-ddc9-410e-8573-5ddd8ac0945a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.280980 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.280978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30327971-ddc9-410e-8573-5ddd8ac0945a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.281192 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.281008    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xszd5\" (UniqueName: \"kubernetes.io/projected/30327971-ddc9-410e-8573-5ddd8ac0945a-kube-api-access-xszd5\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.281192 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.281085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/30327971-ddc9-410e-8573-5ddd8ac0945a-manifests\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.281192 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.281133    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30327971-ddc9-410e-8573-5ddd8ac0945a-tmp\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.338968 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.338920    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:22.381948 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.381923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/30327971-ddc9-410e-8573-5ddd8ac0945a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.382083 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.381955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30327971-ddc9-410e-8573-5ddd8ac0945a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.382083 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.381976    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xszd5\" (UniqueName: \"kubernetes.io/projected/30327971-ddc9-410e-8573-5ddd8ac0945a-kube-api-access-xszd5\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.382083 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.381996    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/30327971-ddc9-410e-8573-5ddd8ac0945a-manifests\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.382083 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.382034    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30327971-ddc9-410e-8573-5ddd8ac0945a-tmp\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.382496 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.382476    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/30327971-ddc9-410e-8573-5ddd8ac0945a-manifests\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.384353 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.384326    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30327971-ddc9-410e-8573-5ddd8ac0945a-tmp\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.384575 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.384558    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30327971-ddc9-410e-8573-5ddd8ac0945a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.384672 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.384652    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/30327971-ddc9-410e-8573-5ddd8ac0945a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.390139 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.390113    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xszd5\" (UniqueName: \"kubernetes.io/projected/30327971-ddc9-410e-8573-5ddd8ac0945a-kube-api-access-xszd5\") pod \"opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb\" (UID: \"30327971-ddc9-410e-8573-5ddd8ac0945a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.432650 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.432629    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:22.439905 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.439876    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:22.541164 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.541134    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:22.590065 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.589688    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"]
Jul 30 14:53:22.594538 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:53:22.594506    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30327971_ddc9_410e_8573_5ddd8ac0945a.slice/crio-eb8080fa57876e30f2819ccf902c4b426eb452c04c353b393c5282e833c40bc7 WatchSource:0}: Error finding container eb8080fa57876e30f2819ccf902c4b426eb452c04c353b393c5282e833c40bc7: Status 404 returned error can't find the container with id eb8080fa57876e30f2819ccf902c4b426eb452c04c353b393c5282e833c40bc7
Jul 30 14:53:22.600082 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.600037    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:22.666988 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.666943    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:22.769293 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.769206    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:22.890874 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:22.890843    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb" event={"ID":"30327971-ddc9-410e-8573-5ddd8ac0945a","Type":"ContainerStarted","Data":"eb8080fa57876e30f2819ccf902c4b426eb452c04c353b393c5282e833c40bc7"}
Jul 30 14:53:26.910210 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:26.910178    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb" event={"ID":"30327971-ddc9-410e-8573-5ddd8ac0945a","Type":"ContainerStarted","Data":"9ac38d918b0d2d28ebd36aca2beb8a5d2638b4b8eb3eff70f7aa8bd6ecb8aecc"}
Jul 30 14:53:26.911790 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:26.911766    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g" event={"ID":"add9f030-6c98-4832-a445-402a4926bdfb","Type":"ContainerStarted","Data":"7de7bdbb92e0358a41859cd888780db28974d1ab0281756774618f9f54c791d1"}
Jul 30 14:53:26.911927 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:26.911821    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:26.947746 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:26.947700    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g" podStartSLOduration=1.664852191 podStartE2EDuration="5.947681786s" totalImagesPullingTime="4.282829595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:53:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:53:26.946225164 +0000 UTC m=+312.619671259" watchObservedRunningTime="2026-07-30 14:53:26.947681786 +0000 UTC m=+312.621127898"
Jul 30 14:53:27.916729 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:27.916695    2582 generic.go:356] "Generic (PLEG): container finished" podID="30327971-ddc9-410e-8573-5ddd8ac0945a" containerID="9ac38d918b0d2d28ebd36aca2beb8a5d2638b4b8eb3eff70f7aa8bd6ecb8aecc" exitCode=0
Jul 30 14:53:27.917117 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:27.916764    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb" event={"ID":"30327971-ddc9-410e-8573-5ddd8ac0945a","Type":"ContainerDied","Data":"9ac38d918b0d2d28ebd36aca2beb8a5d2638b4b8eb3eff70f7aa8bd6ecb8aecc"}
Jul 30 14:53:28.922039 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:28.922001    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb" event={"ID":"30327971-ddc9-410e-8573-5ddd8ac0945a","Type":"ContainerStarted","Data":"a0c6ab163918c1cd5fb627b4bd39d4ec52ec3dd2eaedc83b5391753780abf151"}
Jul 30 14:53:28.922444 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:28.922113    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:28.942347 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:28.942300    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb" podStartSLOduration=2.154816205 podStartE2EDuration="6.942287443s" totalImagesPullingTime="3.719796193s" totalInitContainerRuntime="1.067675045s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:53:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:53:28.941456131 +0000 UTC m=+314.614902254" watchObservedRunningTime="2026-07-30 14:53:28.942287443 +0000 UTC m=+314.615733555"
Jul 30 14:53:36.118445 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:36.118386    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:37.661236 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:37.661200    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:37.839373 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:37.839343    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:37.903993 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:37.903963    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:37.919193 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:37.918921    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b696b7c-fj29g"
Jul 30 14:53:39.783119 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:39.783082    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:39.885639 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:39.885609    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:39.927845 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:39.927815    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb"
Jul 30 14:53:41.501525 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:41.501469    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:41.616591 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:41.616555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:41.885177 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:41.885072    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:42.302517 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:42.302489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:42.484274 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:42.484242    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:42.585987 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:42.585887    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:42.690913 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:42.690876    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:42.797289 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:42.797252    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:43.043472 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.043420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:43.478336 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.478300    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:43.536228 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.536191    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-76v7f"]
Jul 30 14:53:43.544633 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.544605    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:53:43.546571 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.546556    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 30 14:53:43.549819 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.549798    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75vkh\" (UniqueName: \"kubernetes.io/projected/6c0b168d-4cad-4641-8c8d-d92f50c708b2-kube-api-access-75vkh\") pod \"model-serving-api-54d89bd5fc-76v7f\" (UID: \"6c0b168d-4cad-4641-8c8d-d92f50c708b2\") " pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:53:43.549925 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.549857    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0b168d-4cad-4641-8c8d-d92f50c708b2-tls-certs\") pod \"model-serving-api-54d89bd5fc-76v7f\" (UID: \"6c0b168d-4cad-4641-8c8d-d92f50c708b2\") " pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:53:43.552249 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.552226    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-76v7f"]
Jul 30 14:53:43.555806 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.555783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:43.650343 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.650320    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75vkh\" (UniqueName: \"kubernetes.io/projected/6c0b168d-4cad-4641-8c8d-d92f50c708b2-kube-api-access-75vkh\") pod \"model-serving-api-54d89bd5fc-76v7f\" (UID: \"6c0b168d-4cad-4641-8c8d-d92f50c708b2\") " pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:53:43.650699 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.650382    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0b168d-4cad-4641-8c8d-d92f50c708b2-tls-certs\") pod \"model-serving-api-54d89bd5fc-76v7f\" (UID: \"6c0b168d-4cad-4641-8c8d-d92f50c708b2\") " pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:53:43.650699 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:53:43.650525    2582 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 30 14:53:43.650699 ip-10-0-133-92 kubenswrapper[2582]: E0730 14:53:43.650596    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c0b168d-4cad-4641-8c8d-d92f50c708b2-tls-certs podName:6c0b168d-4cad-4641-8c8d-d92f50c708b2 nodeName:}" failed. No retries permitted until 2026-07-30 14:53:44.150573379 +0000 UTC m=+329.824019495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6c0b168d-4cad-4641-8c8d-d92f50c708b2-tls-certs") pod "model-serving-api-54d89bd5fc-76v7f" (UID: "6c0b168d-4cad-4641-8c8d-d92f50c708b2") : secret "model-serving-api-tls" not found
Jul 30 14:53:43.659810 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.659787    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75vkh\" (UniqueName: \"kubernetes.io/projected/6c0b168d-4cad-4641-8c8d-d92f50c708b2-kube-api-access-75vkh\") pod \"model-serving-api-54d89bd5fc-76v7f\" (UID: \"6c0b168d-4cad-4641-8c8d-d92f50c708b2\") " pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:53:43.738817 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:43.738750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:44.154375 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:44.154285    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0b168d-4cad-4641-8c8d-d92f50c708b2-tls-certs\") pod \"model-serving-api-54d89bd5fc-76v7f\" (UID: \"6c0b168d-4cad-4641-8c8d-d92f50c708b2\") " pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:53:44.156851 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:44.156829    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0b168d-4cad-4641-8c8d-d92f50c708b2-tls-certs\") pod \"model-serving-api-54d89bd5fc-76v7f\" (UID: \"6c0b168d-4cad-4641-8c8d-d92f50c708b2\") " pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:53:44.455551 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:44.455517    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:53:44.519522 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:44.519480    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:44.586415 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:44.586370    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-76v7f"]
Jul 30 14:53:44.589279 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:53:44.589254    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c0b168d_4cad_4641_8c8d_d92f50c708b2.slice/crio-504c7822d1302ba2def958a82d3a5d4bd310de2afe748dfc58697e227950c378 WatchSource:0}: Error finding container 504c7822d1302ba2def958a82d3a5d4bd310de2afe748dfc58697e227950c378: Status 404 returned error can't find the container with id 504c7822d1302ba2def958a82d3a5d4bd310de2afe748dfc58697e227950c378
Jul 30 14:53:44.977073 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:44.977038    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-76v7f" event={"ID":"6c0b168d-4cad-4641-8c8d-d92f50c708b2","Type":"ContainerStarted","Data":"504c7822d1302ba2def958a82d3a5d4bd310de2afe748dfc58697e227950c378"}
Jul 30 14:53:46.985642 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:46.985608    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-76v7f" event={"ID":"6c0b168d-4cad-4641-8c8d-d92f50c708b2","Type":"ContainerStarted","Data":"6da993017e2ea8fe5969956660ac8dbd66818905771fc481b6c477dcfb353709"}
Jul 30 14:53:46.986101 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:46.985717    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:53:47.002146 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:47.002101    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-76v7f" podStartSLOduration=2.183209846 podStartE2EDuration="4.002088719s" totalImagesPullingTime="1.818878873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:53:47.000671787 +0000 UTC m=+332.674117916" watchObservedRunningTime="2026-07-30 14:53:47.002088719 +0000 UTC m=+332.675534881"
Jul 30 14:53:49.030952 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:49.030904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:49.131635 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:49.131604    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:49.718304 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:49.718276    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:49.860020 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:49.859988    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:49.959443 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:49.959411    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:51.776709 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.776676    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"]
Jul 30 14:53:51.779765 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.779750    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.782161 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.782143    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 30 14:53:51.782520 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.782502    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 30 14:53:51.782613 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.782596    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 30 14:53:51.782715 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.782698    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 14:53:51.783711 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.783695    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-njlc8\""
Jul 30 14:53:51.793001 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.792980    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"]
Jul 30 14:53:51.807174 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.807147    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d69921a-091f-4898-98be-df24e8bb429f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.807256 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.807176    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d69921a-091f-4898-98be-df24e8bb429f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.807256 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.807220    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d69921a-091f-4898-98be-df24e8bb429f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.807256 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.807237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d69921a-091f-4898-98be-df24e8bb429f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.807352 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.807256    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d69921a-091f-4898-98be-df24e8bb429f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.807352 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.807301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d69921a-091f-4898-98be-df24e8bb429f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.807352 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.807321    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm9ms\" (UniqueName: \"kubernetes.io/projected/8d69921a-091f-4898-98be-df24e8bb429f-kube-api-access-zm9ms\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.908349 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.908327    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d69921a-091f-4898-98be-df24e8bb429f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.908510 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.908362    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm9ms\" (UniqueName: \"kubernetes.io/projected/8d69921a-091f-4898-98be-df24e8bb429f-kube-api-access-zm9ms\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.908510 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.908455    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d69921a-091f-4898-98be-df24e8bb429f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.908510 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.908485    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d69921a-091f-4898-98be-df24e8bb429f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.908676 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.908515    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d69921a-091f-4898-98be-df24e8bb429f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.908676 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.908624    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d69921a-091f-4898-98be-df24e8bb429f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.908676 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.908669    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d69921a-091f-4898-98be-df24e8bb429f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.909239 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.909176    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d69921a-091f-4898-98be-df24e8bb429f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.910973 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.910947    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d69921a-091f-4898-98be-df24e8bb429f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.911082 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.911063    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d69921a-091f-4898-98be-df24e8bb429f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.911353 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.911337    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d69921a-091f-4898-98be-df24e8bb429f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.911443 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.911369    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d69921a-091f-4898-98be-df24e8bb429f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.920787 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.920762    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm9ms\" (UniqueName: \"kubernetes.io/projected/8d69921a-091f-4898-98be-df24e8bb429f-kube-api-access-zm9ms\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:51.922380 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:51.922359    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d69921a-091f-4898-98be-df24e8bb429f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-s4qms\" (UID: \"8d69921a-091f-4898-98be-df24e8bb429f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:52.088953 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:52.088861    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:52.221730 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:52.221688    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"]
Jul 30 14:53:52.224998 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:53:52.224973    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d69921a_091f_4898_98be_df24e8bb429f.slice/crio-30ec3c19bd78d829a6b24d921646c42fc5d696410954e0a53dc4ed35961d7738 WatchSource:0}: Error finding container 30ec3c19bd78d829a6b24d921646c42fc5d696410954e0a53dc4ed35961d7738: Status 404 returned error can't find the container with id 30ec3c19bd78d829a6b24d921646c42fc5d696410954e0a53dc4ed35961d7738
Jul 30 14:53:53.009743 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:53.009701    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms" event={"ID":"8d69921a-091f-4898-98be-df24e8bb429f","Type":"ContainerStarted","Data":"30ec3c19bd78d829a6b24d921646c42fc5d696410954e0a53dc4ed35961d7738"}
Jul 30 14:53:54.343882 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:54.343846    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:54.446871 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:54.446837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:54.548864 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:54.548831    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:54.570217 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:54.570184    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 14:53:54.570336 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:54.570264    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 14:53:54.731337 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:54.731263    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:54.832834 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:54.832796    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:55.019795 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:55.019758    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms" event={"ID":"8d69921a-091f-4898-98be-df24e8bb429f","Type":"ContainerStarted","Data":"08917e474a9abe40819c77f525d3b7270dee2c761a832dc0c881b0e1c66ff0d3"}
Jul 30 14:53:55.019984 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:55.019966    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:55.021583 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:55.021552    2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-s4qms container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 14:53:55.021670 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:55.021611    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms" podUID="8d69921a-091f-4898-98be-df24e8bb429f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 14:53:55.090113 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:55.090059    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms" podStartSLOduration=1.7468673 podStartE2EDuration="4.090022055s" totalImagesPullingTime="2.343154755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:53:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:53:55.086780091 +0000 UTC m=+340.760226228" watchObservedRunningTime="2026-07-30 14:53:55.090022055 +0000 UTC m=+340.763468169"
Jul 30 14:53:55.859667 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:55.859640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:55.958595 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:55.958557    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:56.024661 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:56.024634    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-s4qms"
Jul 30 14:53:56.030300 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:56.030274    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:53:57.993506 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:53:57.993473    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-76v7f"
Jul 30 14:54:01.706770 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:54:01.706736    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:54:01.767571 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:54:01.767539    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:54:43.164850 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:54:43.164776    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:54:43.280025 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:54:43.279992    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:54:43.450335 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:54:43.450303    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:54:44.128870 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:54:44.128836    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:54:44.343712 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:54:44.343677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:12.263752 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:12.263717    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:14.057551 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:14.057518    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:14.113024 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:14.112992    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:15.467874 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:15.467840    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:15.564891 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:15.564853    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:20.297017 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:20.296984    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:20.877052 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:20.877024    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:22.908073 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:22.908036    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:24.489303 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:24.489273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:24.679895 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:24.679862    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:24.967333 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:24.967288    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:25.069793 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:25.069759    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:25.166299 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:25.166242    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:25.373382 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:25.373299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:25.486831 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:25.486796    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:27.351984 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:27.351960    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:28.108713 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:28.108673    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:28.406756 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:28.406676    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:28.496219 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:28.496181    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:28.681673 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:28.681646    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:28.782143 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:28.782114    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:31.865485 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:31.865458    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:31.934450 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:31.934422    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:40.701209 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:40.701129    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:40.792589 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:40.792547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:43.149654 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:43.149617    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:43.249876 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:43.249842    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:47.409102 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:47.409026    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:47.509988 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:47.509956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:49.690505 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:49.690474    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:49.787503 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:49.787471    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:52.858240 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:52.858203    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:58.386566 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:58.386539    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:55:58.487855 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:55:58.487826    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:00.858690 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:00.858652    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:00.960462 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:00.960425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:05.325012 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:05.324976    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:05.424878 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:05.424837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:08.191423 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:08.191366    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:08.291763 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:08.291729    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:16.596948 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:16.596913    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:16.970532 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:16.970506    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:17.071554 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:17.071525    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:17.994277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:17.994246    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:18.075600 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:18.075573    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:18.412522 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:18.412443    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:18.472597 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:18.472569    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:21.535299 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:21.535270    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:21.722024 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:21.721990    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:22.195157 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:22.195125    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:22.295953 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:22.295915    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:23.907487 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:23.907455    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:23.969782 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:23.969752    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:24.252968 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.252937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:24.451424 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.451372    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:24.599248 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.599171    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:24.676080 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.676050    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:56:24.679182 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.679164    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng"
Jul 30 14:56:24.681454 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.681434    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 14:56:24.681534 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.681455    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 14:56:24.681534 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.681485    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 14:56:24.681800 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.681787    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k6jbg\""
Jul 30 14:56:24.689865 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.689845    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:56:24.704889 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.704865    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:56:24.814735 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.814707    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnfb7\" (UniqueName: \"kubernetes.io/projected/b1d79f03-bdb7-4a7d-9261-232438552ada-kube-api-access-rnfb7\") pod \"limitador-limitador-54d6d988d5-nrsng\" (UID: \"b1d79f03-bdb7-4a7d-9261-232438552ada\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng"
Jul 30 14:56:24.814899 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.814759    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b1d79f03-bdb7-4a7d-9261-232438552ada-config-file\") pod \"limitador-limitador-54d6d988d5-nrsng\" (UID: \"b1d79f03-bdb7-4a7d-9261-232438552ada\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng"
Jul 30 14:56:24.915747 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.915667    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b1d79f03-bdb7-4a7d-9261-232438552ada-config-file\") pod \"limitador-limitador-54d6d988d5-nrsng\" (UID: \"b1d79f03-bdb7-4a7d-9261-232438552ada\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng"
Jul 30 14:56:24.915747 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.915718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnfb7\" (UniqueName: \"kubernetes.io/projected/b1d79f03-bdb7-4a7d-9261-232438552ada-kube-api-access-rnfb7\") pod \"limitador-limitador-54d6d988d5-nrsng\" (UID: \"b1d79f03-bdb7-4a7d-9261-232438552ada\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng"
Jul 30 14:56:24.916277 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.916259    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b1d79f03-bdb7-4a7d-9261-232438552ada-config-file\") pod \"limitador-limitador-54d6d988d5-nrsng\" (UID: \"b1d79f03-bdb7-4a7d-9261-232438552ada\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng"
Jul 30 14:56:24.923980 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.923952    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnfb7\" (UniqueName: \"kubernetes.io/projected/b1d79f03-bdb7-4a7d-9261-232438552ada-kube-api-access-rnfb7\") pod \"limitador-limitador-54d6d988d5-nrsng\" (UID: \"b1d79f03-bdb7-4a7d-9261-232438552ada\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng"
Jul 30 14:56:24.988978 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:24.988944    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng"
Jul 30 14:56:25.126359 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:25.126331    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:56:25.130756 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:56:25.130729    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d79f03_bdb7_4a7d_9261_232438552ada.slice/crio-e2307cbc1e5431808a4ffa6579ed8214e53d6d4be2f715fd836846324cd1a106 WatchSource:0}: Error finding container e2307cbc1e5431808a4ffa6579ed8214e53d6d4be2f715fd836846324cd1a106: Status 404 returned error can't find the container with id e2307cbc1e5431808a4ffa6579ed8214e53d6d4be2f715fd836846324cd1a106
Jul 30 14:56:25.153812 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:25.153781    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:25.256291 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:25.256264    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:25.589504 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:25.589416    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng" event={"ID":"b1d79f03-bdb7-4a7d-9261-232438552ada","Type":"ContainerStarted","Data":"e2307cbc1e5431808a4ffa6579ed8214e53d6d4be2f715fd836846324cd1a106"}
Jul 30 14:56:28.602355 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:28.602317    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng" event={"ID":"b1d79f03-bdb7-4a7d-9261-232438552ada","Type":"ContainerStarted","Data":"d364c4cdbcd8cad978157315ba9d07725f2a059ce64c73912582d068f6adee34"}
Jul 30 14:56:28.602813 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:28.602444    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng"
Jul 30 14:56:28.626002 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:28.625955    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng" podStartSLOduration=2.110532173 podStartE2EDuration="4.625940799s" totalImagesPullingTime="2.515408626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:56:28.620696429 +0000 UTC m=+494.294142540" watchObservedRunningTime="2026-07-30 14:56:28.625940799 +0000 UTC m=+494.299386912"
Jul 30 14:56:39.606871 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:39.606843    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nrsng"
Jul 30 14:56:45.575962 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:45.575931    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:45.676146 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:45.676115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:46.072246 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:46.072215    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:58.536930 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:58.536896    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:56:58.826512 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:56:58.826434    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:05.334827 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:05.334795    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:05.432157 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:05.432126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:05.717557 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:05.717528    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:08.988605 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:08.988577    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:09.102654 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:09.102623    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:40.611047 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:40.611006    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:40.705147 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:40.705110    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:41.417894 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:41.417865    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:41.518378 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:41.518350    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:42.320592 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:42.320556    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:42.582328 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:42.582247    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:43.014047 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:43.014018    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:43.114353 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:43.114326    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:43.233794 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:43.233758    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:57:52.498676 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:52.498637    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:57:52.612210 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:57:52.612180    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:58:12.061460 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:58:12.061419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:58:12.337592 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:58:12.337516    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:58:15.848062 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:58:15.848027    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:58:19.724571 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:58:19.724537    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:58:33.443784 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:58:33.443751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:58:34.219422 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:58:34.219373    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:58:42.943203 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:58:42.943121    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:58:44.123038 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:58:44.123007    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:58:57.868614 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:58:57.868580    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:58:59.434968 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:58:59.434931    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:59:05.918526 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:05.918491    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:59:22.837331 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:22.837298    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:59:25.628759 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:25.628727    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:59:36.830260 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.830210    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"]
Jul 30 14:59:36.833816 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.833795    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.836776 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.836751    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-756b8\""
Jul 30 14:59:36.836880 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.836786    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 14:59:36.836880 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.836795    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 14:59:36.836880 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.836794    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 30 14:59:36.843665 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.843642    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"]
Jul 30 14:59:36.863571 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.863545    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 14:59:36.873198 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.873178    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.873292 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.873208    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.873292 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.873232    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.873380 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.873292    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.873380 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.873324    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.873380 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.873340    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsgnr\" (UniqueName: \"kubernetes.io/projected/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-kube-api-access-xsgnr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.974151 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.974123    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.974238 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.974158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.974238 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.974182    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.974238 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.974207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.974347 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.974242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.974347 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.974258    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsgnr\" (UniqueName: \"kubernetes.io/projected/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-kube-api-access-xsgnr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.974652 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.974631    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.974652 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.974649    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.974779 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.974696    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.976516 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.976495    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.976747 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.976731    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:36.982448 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:36.982423    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsgnr\" (UniqueName: \"kubernetes.io/projected/cb107ec1-19ae-4acb-9e2f-ff6a58277c9b-kube-api-access-xsgnr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt\" (UID: \"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:37.143044 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:37.142959    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:37.273223 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:37.273194    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"]
Jul 30 14:59:37.276338 ip-10-0-133-92 kubenswrapper[2582]: W0730 14:59:37.276313    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb107ec1_19ae_4acb_9e2f_ff6a58277c9b.slice/crio-f7282660adb8b74e33f0563fe86549ad6e5c76accd1df20c47b17a6c113f0cf6 WatchSource:0}: Error finding container f7282660adb8b74e33f0563fe86549ad6e5c76accd1df20c47b17a6c113f0cf6: Status 404 returned error can't find the container with id f7282660adb8b74e33f0563fe86549ad6e5c76accd1df20c47b17a6c113f0cf6
Jul 30 14:59:37.277641 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:37.277624    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 14:59:37.320098 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:37.320067    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt" event={"ID":"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b","Type":"ContainerStarted","Data":"f7282660adb8b74e33f0563fe86549ad6e5c76accd1df20c47b17a6c113f0cf6"}
Jul 30 14:59:38.325791 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:38.325756    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt" event={"ID":"cb107ec1-19ae-4acb-9e2f-ff6a58277c9b","Type":"ContainerStarted","Data":"781b421dae868d0e27459afe2b81aeef13f2bd25504ce2712bf4d86d13f8acd2"}
Jul 30 14:59:38.345806 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:38.345763    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt" podStartSLOduration=1.383858123 podStartE2EDuration="2.345747078s" totalImagesPullingTime="961.888955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 14:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 14:59:38.344776592 +0000 UTC m=+684.018222724" watchObservedRunningTime="2026-07-30 14:59:38.345747078 +0000 UTC m=+684.019193191"
Jul 30 14:59:39.232224 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:39.232182    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 14:59:47.143388 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:47.143351    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:47.143861 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:47.143668    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:47.155868 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:47.155842    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 14:59:47.371754 ip-10-0-133-92 kubenswrapper[2582]: I0730 14:59:47.371723    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt"
Jul 30 15:01:12.086012 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:01:12.085968    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:01:12.576932 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:01:12.576892    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:01:12.718990 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:01:12.718955    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:01:13.871714 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:01:13.871678    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:01:14.257265 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:01:14.257232    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:01:42.827004 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:01:42.826914    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:01:43.377549 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:01:43.377513    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:01:54.724473 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:01:54.724435    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:02:04.427520 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:02:04.427484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:02:37.598572 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:02:37.598535    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:03:20.455522 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:03:20.455486    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:03:21.615203 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:03:21.615161    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:03:21.720757 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:03:21.720720    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:03:42.226599 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:03:42.226565    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:03:44.766245 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:03:44.766203    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:03:53.933083 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:03:53.933048    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:04:06.723546 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:06.723514    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:06.782463 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:06.782427    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:37.107147 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:37.107064    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:37.202809 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:37.202775    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:37.362383 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:37.362300    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:43.167051 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:43.167005    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:43.228858 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:43.228824    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:43.315200 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:43.315135    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:43.404256 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:43.404228    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:43.554557 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:43.554527    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:49.814497 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:49.814463    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:04:49.913296 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:04:49.913261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:05:07.324854 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:05:07.324811    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:05:16.826055 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:05:16.826021    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:05:31.825253 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:05:31.825214    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:05:40.826090 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:05:40.826057    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:05:52.729450 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:05:52.729418    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:06:02.329647 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:06:02.329612    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:06:13.227974 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:06:13.227941    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:06:22.141297 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:06:22.141264    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:06:44.425431 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:06:44.425382    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:07:07.321858 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:07:07.321822    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:07:28.924684 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:07:28.924649    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:07:42.729694 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:07:42.729601    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:07:58.030319 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:07:58.030285    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:08:16.828995 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:08:16.828955    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:08:32.532155 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:08:32.532125    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:08:42.324699 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:08:42.324589    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:08:49.241712 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:08:49.241674    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:08:52.427217 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:08:52.427180    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:09:03.828462 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:09:03.828424    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:09:14.820899 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:09:14.816270    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:09:24.816406 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:09:24.816359    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:09:36.326451 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:09:36.326410    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:10:33.502920 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:10:33.502880    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:10:33.565341 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:10:33.565307    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:11:10.615879 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:11:10.615845    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:11:52.029121 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:11:52.029084    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:11:59.822154 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:11:59.822113    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:12:26.226998 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:12:26.226923    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:13:27.121082 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:13:27.121045    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:13:30.589911 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:13:30.589876    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:13:36.823935 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:13:36.823865    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:13:44.924865 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:13:44.924831    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:13:55.934339 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:13:55.934303    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:14:07.329144 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:14:07.329108    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:14:16.431471 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:14:16.431434    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:14:28.119892 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:14:28.119858    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:14:38.819920 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:14:38.819888    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:14:43.010469 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:14:43.010433    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:14:49.322560 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:14:49.322524    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:15:09.523385 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:09.523287    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:15:17.625033 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:17.624995    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:15:27.072795 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:27.072763    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:15:30.026230 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:30.026196    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:15:49.019171 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.019143    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"]
Jul 30 15:15:49.022196 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.022177    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.024047 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.024029    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-6405f46e-kserve-self-signed-certs\""
Jul 30 15:15:49.030503 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.030484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"]
Jul 30 15:15:49.060612 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.060586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:15:49.149246 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.149198    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-home\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.149471 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.149262    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tmp-dir\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.149471 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.149287    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-dshm\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.149471 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.149337    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-model-cache\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.149471 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.149450    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp9s4\" (UniqueName: \"kubernetes.io/projected/e06ea40b-0831-4447-87ec-39e55d5c4ddf-kube-api-access-sp9s4\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.149636 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.149478    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tls-certs\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.250494 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.250456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-model-cache\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.250696 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.250514    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp9s4\" (UniqueName: \"kubernetes.io/projected/e06ea40b-0831-4447-87ec-39e55d5c4ddf-kube-api-access-sp9s4\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.250696 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.250619    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tls-certs\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.250696 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.250692    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-home\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.250856 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.250734    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tmp-dir\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.250856 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.250763    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-dshm\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.250938 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.250905    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-model-cache\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.251074 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.251051    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tmp-dir\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.251074 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.251066    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-home\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.253001 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.252985    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-dshm\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.253245 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.253230    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tls-certs\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.257268 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.257238    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp9s4\" (UniqueName: \"kubernetes.io/projected/e06ea40b-0831-4447-87ec-39e55d5c4ddf-kube-api-access-sp9s4\") pod \"e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") " pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.332350 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.332267    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:49.458168 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:49.458131    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"]
Jul 30 15:15:49.460810 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:15:49.460786    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode06ea40b_0831_4447_87ec_39e55d5c4ddf.slice/crio-76c158e0539589b658e81537cbbbb4cff1834f1b84595353c87c538d0b73470d WatchSource:0}: Error finding container 76c158e0539589b658e81537cbbbb4cff1834f1b84595353c87c538d0b73470d: Status 404 returned error can't find the container with id 76c158e0539589b658e81537cbbbb4cff1834f1b84595353c87c538d0b73470d
Jul 30 15:15:50.121272 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:50.121234    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm" event={"ID":"e06ea40b-0831-4447-87ec-39e55d5c4ddf","Type":"ContainerStarted","Data":"0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737"}
Jul 30 15:15:50.121701 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:50.121282    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm" event={"ID":"e06ea40b-0831-4447-87ec-39e55d5c4ddf","Type":"ContainerStarted","Data":"76c158e0539589b658e81537cbbbb4cff1834f1b84595353c87c538d0b73470d"}
Jul 30 15:15:50.137932 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:50.137871    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm" podStartSLOduration=1.137851721 podStartE2EDuration="1.137851721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:15:50.135765966 +0000 UTC m=+1655.809212081" watchObservedRunningTime="2026-07-30 15:15:50.137851721 +0000 UTC m=+1655.811297850"
Jul 30 15:15:52.918993 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:52.918961    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:15:59.332910 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:59.332875    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:59.333326 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:59.332928    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:15:59.345441 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:15:59.345416    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:16:00.171869 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:00.171839    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:16:14.616572 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:14.616543    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:16:16.218493 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:16.218457    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"]
Jul 30 15:16:16.229980 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:16.229946    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:16:17.428962 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:17.428930    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:16:31.238048 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.238004    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm" podUID="e06ea40b-0831-4447-87ec-39e55d5c4ddf" containerName="main" containerID="cri-o://0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737" gracePeriod=45
Jul 30 15:16:31.480720 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.480693    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:16:31.579075 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.578980    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-model-cache\") pod \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") "
Jul 30 15:16:31.579075 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.579021    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp9s4\" (UniqueName: \"kubernetes.io/projected/e06ea40b-0831-4447-87ec-39e55d5c4ddf-kube-api-access-sp9s4\") pod \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") "
Jul 30 15:16:31.579075 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.579053    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tls-certs\") pod \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") "
Jul 30 15:16:31.579352 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.579085    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-dshm\") pod \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") "
Jul 30 15:16:31.579352 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.579118    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-home\") pod \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") "
Jul 30 15:16:31.579352 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.579134    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tmp-dir\") pod \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\" (UID: \"e06ea40b-0831-4447-87ec-39e55d5c4ddf\") "
Jul 30 15:16:31.579352 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.579310    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-model-cache" (OuterVolumeSpecName: "model-cache") pod "e06ea40b-0831-4447-87ec-39e55d5c4ddf" (UID: "e06ea40b-0831-4447-87ec-39e55d5c4ddf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:16:31.579575 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.579491    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e06ea40b-0831-4447-87ec-39e55d5c4ddf" (UID: "e06ea40b-0831-4447-87ec-39e55d5c4ddf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:16:31.579659 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.579630    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-home" (OuterVolumeSpecName: "home") pod "e06ea40b-0831-4447-87ec-39e55d5c4ddf" (UID: "e06ea40b-0831-4447-87ec-39e55d5c4ddf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:16:31.581542 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.581521    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e06ea40b-0831-4447-87ec-39e55d5c4ddf" (UID: "e06ea40b-0831-4447-87ec-39e55d5c4ddf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:16:31.581542 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.581536    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-dshm" (OuterVolumeSpecName: "dshm") pod "e06ea40b-0831-4447-87ec-39e55d5c4ddf" (UID: "e06ea40b-0831-4447-87ec-39e55d5c4ddf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:16:31.581662 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.581545    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e06ea40b-0831-4447-87ec-39e55d5c4ddf-kube-api-access-sp9s4" (OuterVolumeSpecName: "kube-api-access-sp9s4") pod "e06ea40b-0831-4447-87ec-39e55d5c4ddf" (UID: "e06ea40b-0831-4447-87ec-39e55d5c4ddf"). InnerVolumeSpecName "kube-api-access-sp9s4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:16:31.680371 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.680316    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tls-certs\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:16:31.680371 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.680363    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-dshm\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:16:31.680371 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.680372    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-home\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:16:31.680371 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.680381    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-tmp-dir\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:16:31.680371 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.680389    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e06ea40b-0831-4447-87ec-39e55d5c4ddf-model-cache\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:16:31.680685 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:31.680419    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp9s4\" (UniqueName: \"kubernetes.io/projected/e06ea40b-0831-4447-87ec-39e55d5c4ddf-kube-api-access-sp9s4\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:16:32.292907 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:32.292864    2582 generic.go:356] "Generic (PLEG): container finished" podID="e06ea40b-0831-4447-87ec-39e55d5c4ddf" containerID="0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737" exitCode=0
Jul 30 15:16:32.293275 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:32.292926    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"
Jul 30 15:16:32.293275 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:32.292946    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm" event={"ID":"e06ea40b-0831-4447-87ec-39e55d5c4ddf","Type":"ContainerDied","Data":"0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737"}
Jul 30 15:16:32.293275 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:32.292988    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm" event={"ID":"e06ea40b-0831-4447-87ec-39e55d5c4ddf","Type":"ContainerDied","Data":"76c158e0539589b658e81537cbbbb4cff1834f1b84595353c87c538d0b73470d"}
Jul 30 15:16:32.293849 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:32.293835    2582 scope.go:117] "RemoveContainer" containerID="0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737"
Jul 30 15:16:32.302293 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:32.302272    2582 scope.go:117] "RemoveContainer" containerID="0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737"
Jul 30 15:16:32.302602 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:16:32.302582    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737\": container with ID starting with 0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737 not found: ID does not exist" containerID="0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737"
Jul 30 15:16:32.302672 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:32.302609    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737"} err="failed to get container status \"0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737\": rpc error: code = NotFound desc = could not find container \"0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737\": container with ID starting with 0251144602a774cef5f082acd04abf0376b0fd781bc889cd18c2d6de8f5d5737 not found: ID does not exist"
Jul 30 15:16:32.312105 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:32.312082    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"]
Jul 30 15:16:32.317082 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:32.317053    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-6405f46e-kserve-7549dd4847-tvqwm"]
Jul 30 15:16:32.807178 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:32.807137    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e06ea40b-0831-4447-87ec-39e55d5c4ddf" path="/var/lib/kubelet/pods/e06ea40b-0831-4447-87ec-39e55d5c4ddf/volumes"
Jul 30 15:16:33.325988 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:33.325954    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:16:40.420364 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:40.420273    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:16:50.223764 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:50.223730    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:16:57.027206 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:16:57.027166    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:17:12.229441 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:17:12.229386    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:17:23.421915 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:17:23.421881    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:17:33.421388 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:17:33.421354    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:17:43.325671 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:17:43.325636    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:17:53.324325 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:17:53.324288    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:18:03.820653 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:18:03.820618    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:18:15.728042 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:18:15.727998    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:18:25.120058 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:18:25.120029    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:18:38.017591 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:18:38.017553    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:18:48.819982 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:18:48.819950    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:18:59.226799 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:18:59.226765    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:19:20.928957 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:19:20.928917    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:19:33.322291 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:19:33.322249    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:19:48.918081 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:19:48.918049    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:19:50.595974 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:19:50.595940    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:19:53.338283 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:19:53.338250    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:20:04.021608 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:20:04.021575    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:20:13.218255 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:20:13.218217    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:20:15.330754 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:20:15.330713    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:20:26.621389 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:20:26.621355    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:20:41.321814 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:20:41.321782    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:20:50.922981 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:20:50.922939    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:20:51.986527 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:20:51.986495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:20:55.238720 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:20:55.238684    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:20:55.341991 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:20:55.341965    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:20:55.783249 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:20:55.783214    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:01.125056 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:01.125018    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:21:01.608356 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:01.608323    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:01.664656 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:01.664617    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:01.865122 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:01.865040    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:02.181931 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:02.181906    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:02.279578 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:02.279542    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:07.170642 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:07.170548    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:07.271697 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:07.271666    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:08.429688 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:08.429655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:08.882233 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:08.882203    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:09.569541 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:09.569505    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:12.421675 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:12.421593    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:21:26.619129 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:26.619091    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:32.881609 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:32.881567    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"]
Jul 30 15:21:32.882196 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:32.881944    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e06ea40b-0831-4447-87ec-39e55d5c4ddf" containerName="main"
Jul 30 15:21:32.882196 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:32.881960    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e06ea40b-0831-4447-87ec-39e55d5c4ddf" containerName="main"
Jul 30 15:21:32.882196 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:32.882042    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e06ea40b-0831-4447-87ec-39e55d5c4ddf" containerName="main"
Jul 30 15:21:32.885031 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:32.885009    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"
Jul 30 15:21:32.886935 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:32.886915    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 15:21:32.887045 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:32.886929    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:21:32.891672 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:32.891650    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"]
Jul 30 15:21:33.034461 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:33.034424    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d94a74d2-659a-4214-8795-08fcde98f880-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-624a770e-jwkpj\" (UID: \"d94a74d2-659a-4214-8795-08fcde98f880\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"
Jul 30 15:21:33.135566 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:33.135493    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d94a74d2-659a-4214-8795-08fcde98f880-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-624a770e-jwkpj\" (UID: \"d94a74d2-659a-4214-8795-08fcde98f880\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"
Jul 30 15:21:33.136069 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:33.136050    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d94a74d2-659a-4214-8795-08fcde98f880-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-624a770e-jwkpj\" (UID: \"d94a74d2-659a-4214-8795-08fcde98f880\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"
Jul 30 15:21:33.196648 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:33.196625    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"
Jul 30 15:21:33.322117 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:33.322082    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"]
Jul 30 15:21:33.324905 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:21:33.324874    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd94a74d2_659a_4214_8795_08fcde98f880.slice/crio-99947ccf43afb4f029f2706d7a22b166854e06c60f97cb9b0d1cf48bbffcdea8 WatchSource:0}: Error finding container 99947ccf43afb4f029f2706d7a22b166854e06c60f97cb9b0d1cf48bbffcdea8: Status 404 returned error can't find the container with id 99947ccf43afb4f029f2706d7a22b166854e06c60f97cb9b0d1cf48bbffcdea8
Jul 30 15:21:33.326747 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:33.326733    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 15:21:33.461251 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:33.461219    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj" event={"ID":"d94a74d2-659a-4214-8795-08fcde98f880","Type":"ContainerStarted","Data":"99947ccf43afb4f029f2706d7a22b166854e06c60f97cb9b0d1cf48bbffcdea8"}
Jul 30 15:21:36.475350 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:36.475308    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj" event={"ID":"d94a74d2-659a-4214-8795-08fcde98f880","Type":"ContainerStarted","Data":"68a3387c0609127a14ddb3a38c0a9bc085e87065a788945f1d9efc2e36fa3d20"}
Jul 30 15:21:36.489994 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:36.489944    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj" podStartSLOduration=1.831088105 podStartE2EDuration="4.489930315s" totalImagesPullingTime="2.65884221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:21:36.489206062 +0000 UTC m=+2002.162652178" watchObservedRunningTime="2026-07-30 15:21:36.489930315 +0000 UTC m=+2002.163376454"
Jul 30 15:21:37.238295 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:37.238264    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:37.330565 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:37.330534    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:37.480343 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:37.480304    2582 generic.go:356] "Generic (PLEG): container finished" podID="d94a74d2-659a-4214-8795-08fcde98f880" containerID="68a3387c0609127a14ddb3a38c0a9bc085e87065a788945f1d9efc2e36fa3d20" exitCode=0
Jul 30 15:21:37.480801 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:37.480371    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj" event={"ID":"d94a74d2-659a-4214-8795-08fcde98f880","Type":"ContainerDied","Data":"68a3387c0609127a14ddb3a38c0a9bc085e87065a788945f1d9efc2e36fa3d20"}
Jul 30 15:21:38.605540 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:38.605516    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"
Jul 30 15:21:38.683180 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:38.683150    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d94a74d2-659a-4214-8795-08fcde98f880-maas-api-service-ca\") pod \"d94a74d2-659a-4214-8795-08fcde98f880\" (UID: \"d94a74d2-659a-4214-8795-08fcde98f880\") "
Jul 30 15:21:38.683548 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:38.683520    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d94a74d2-659a-4214-8795-08fcde98f880-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d94a74d2-659a-4214-8795-08fcde98f880" (UID: "d94a74d2-659a-4214-8795-08fcde98f880"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:21:38.784428 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:38.784352    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d94a74d2-659a-4214-8795-08fcde98f880-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:21:39.490877 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:39.490845    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj" event={"ID":"d94a74d2-659a-4214-8795-08fcde98f880","Type":"ContainerDied","Data":"99947ccf43afb4f029f2706d7a22b166854e06c60f97cb9b0d1cf48bbffcdea8"}
Jul 30 15:21:39.491044 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:39.490897    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99947ccf43afb4f029f2706d7a22b166854e06c60f97cb9b0d1cf48bbffcdea8"
Jul 30 15:21:39.491044 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:39.490860    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"
Jul 30 15:21:48.843544 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:48.843514    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:48.916316 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:48.916282    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:49.004750 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:49.004719    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"]
Jul 30 15:21:49.008872 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:49.008849    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-624a770e-jwkpj"]
Jul 30 15:21:49.012774 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:49.012742    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:49.180345 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:49.180266    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:49.569095 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:49.569061    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:49.723619 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:49.723587    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:49.828204 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:49.828134    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:49.968606 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:49.968573    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:50.299539 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:50.299513    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:50.651354 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:50.651266    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:50.806567 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:50.806535    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d94a74d2-659a-4214-8795-08fcde98f880" path="/var/lib/kubelet/pods/d94a74d2-659a-4214-8795-08fcde98f880/volumes"
Jul 30 15:21:52.555154 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:52.555119    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:52.791926 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:52.791895    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:53.133723 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:53.133692    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:53.227038 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:53.227010    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:21:53.264256 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:53.264227    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:21:53.364581 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:21:53.364525    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:04.022260 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:04.022223    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:22:17.642517 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.642483    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"]
Jul 30 15:22:17.642887 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.642767    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d94a74d2-659a-4214-8795-08fcde98f880" containerName="revoke-keys"
Jul 30 15:22:17.642887 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.642779    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d94a74d2-659a-4214-8795-08fcde98f880" containerName="revoke-keys"
Jul 30 15:22:17.642887 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.642837    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d94a74d2-659a-4214-8795-08fcde98f880" containerName="revoke-keys"
Jul 30 15:22:17.645679 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.645663    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"
Jul 30 15:22:17.647644 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.647616    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:22:17.647772 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.647652    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 15:22:17.652974 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.652954    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"]
Jul 30 15:22:17.790594 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.790558    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff81622-15a2-40f8-9611-5ab1a9206b88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm\" (UID: \"7ff81622-15a2-40f8-9611-5ab1a9206b88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"
Jul 30 15:22:17.891542 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.891504    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff81622-15a2-40f8-9611-5ab1a9206b88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm\" (UID: \"7ff81622-15a2-40f8-9611-5ab1a9206b88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"
Jul 30 15:22:17.892104 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.892082    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff81622-15a2-40f8-9611-5ab1a9206b88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm\" (UID: \"7ff81622-15a2-40f8-9611-5ab1a9206b88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"
Jul 30 15:22:17.955928 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:17.955885    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"
Jul 30 15:22:18.082428 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:18.082373    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"]
Jul 30 15:22:18.085486 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:22:18.085449    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff81622_15a2_40f8_9611_5ab1a9206b88.slice/crio-ab684219fc19c888c6279a5acd934bb6547f84df8f41f8d6bf5afe452430d5b9 WatchSource:0}: Error finding container ab684219fc19c888c6279a5acd934bb6547f84df8f41f8d6bf5afe452430d5b9: Status 404 returned error can't find the container with id ab684219fc19c888c6279a5acd934bb6547f84df8f41f8d6bf5afe452430d5b9
Jul 30 15:22:18.642495 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:18.642375    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm" event={"ID":"7ff81622-15a2-40f8-9611-5ab1a9206b88","Type":"ContainerStarted","Data":"969c4d953e23317801693bbcc58795af40dfc2fbeddbdfd82f7c5d1bc9f65d60"}
Jul 30 15:22:18.642495 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:18.642477    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm" event={"ID":"7ff81622-15a2-40f8-9611-5ab1a9206b88","Type":"ContainerStarted","Data":"ab684219fc19c888c6279a5acd934bb6547f84df8f41f8d6bf5afe452430d5b9"}
Jul 30 15:22:18.657373 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:18.657329    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm" podStartSLOduration=1.657311961 podStartE2EDuration="1.657311961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:22:18.655853318 +0000 UTC m=+2044.329299432" watchObservedRunningTime="2026-07-30 15:22:18.657311961 +0000 UTC m=+2044.330758078"
Jul 30 15:22:19.647221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:19.647183    2582 generic.go:356] "Generic (PLEG): container finished" podID="7ff81622-15a2-40f8-9611-5ab1a9206b88" containerID="969c4d953e23317801693bbcc58795af40dfc2fbeddbdfd82f7c5d1bc9f65d60" exitCode=0
Jul 30 15:22:19.647608 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:19.647246    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm" event={"ID":"7ff81622-15a2-40f8-9611-5ab1a9206b88","Type":"ContainerDied","Data":"969c4d953e23317801693bbcc58795af40dfc2fbeddbdfd82f7c5d1bc9f65d60"}
Jul 30 15:22:20.778516 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:20.778492    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"
Jul 30 15:22:20.916373 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:20.916280    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff81622-15a2-40f8-9611-5ab1a9206b88-maas-api-service-ca\") pod \"7ff81622-15a2-40f8-9611-5ab1a9206b88\" (UID: \"7ff81622-15a2-40f8-9611-5ab1a9206b88\") "
Jul 30 15:22:20.916662 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:20.916639    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff81622-15a2-40f8-9611-5ab1a9206b88-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ff81622-15a2-40f8-9611-5ab1a9206b88" (UID: "7ff81622-15a2-40f8-9611-5ab1a9206b88"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:22:21.016766 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:21.016733    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff81622-15a2-40f8-9611-5ab1a9206b88-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:22:21.657818 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:21.657787    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"
Jul 30 15:22:21.657818 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:21.657801    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm" event={"ID":"7ff81622-15a2-40f8-9611-5ab1a9206b88","Type":"ContainerDied","Data":"ab684219fc19c888c6279a5acd934bb6547f84df8f41f8d6bf5afe452430d5b9"}
Jul 30 15:22:21.658023 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:21.657849    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab684219fc19c888c6279a5acd934bb6547f84df8f41f8d6bf5afe452430d5b9"
Jul 30 15:22:23.993925 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:23.993891    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:24.098082 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:24.098045    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:24.172159 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:24.172120    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"]
Jul 30 15:22:24.174074 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:24.174039    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d54eb26-j25wm"]
Jul 30 15:22:24.176673 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:24.176652    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:24.450642 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:24.450605    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:24.550445 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:24.550387    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:24.650521 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:24.650488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:24.806757 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:24.806683    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ff81622-15a2-40f8-9611-5ab1a9206b88" path="/var/lib/kubelet/pods/7ff81622-15a2-40f8-9611-5ab1a9206b88/volumes"
Jul 30 15:22:24.845364 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:24.845333    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:25.197361 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:25.197332    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:25.456149 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:25.456062    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:27.628994 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:27.628959    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:27.727662 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:27.727631    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:28.024607 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:28.024577    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:22:28.802179 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:28.802142    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:29.295048 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:29.295018    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:29.391531 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:29.391492    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:29.481748 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:29.481715    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:38.319236 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:38.319152    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:22:53.612484 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.612451    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"]
Jul 30 15:22:53.612881 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.612717    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ff81622-15a2-40f8-9611-5ab1a9206b88" containerName="revoke-keys"
Jul 30 15:22:53.612881 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.612727    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ff81622-15a2-40f8-9611-5ab1a9206b88" containerName="revoke-keys"
Jul 30 15:22:53.612881 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.612788    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ff81622-15a2-40f8-9611-5ab1a9206b88" containerName="revoke-keys"
Jul 30 15:22:53.615565 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.615550    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"
Jul 30 15:22:53.617535 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.617514    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 15:22:53.617625 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.617587    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:22:53.623245 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.623222    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"]
Jul 30 15:22:53.763603 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.763570    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-73521211-rqjhl\" (UID: \"5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"
Jul 30 15:22:53.865065 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.864975    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-73521211-rqjhl\" (UID: \"5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"
Jul 30 15:22:53.865610 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.865591    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-73521211-rqjhl\" (UID: \"5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"
Jul 30 15:22:53.926921 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:53.926888    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"
Jul 30 15:22:54.054804 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:54.054769    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"]
Jul 30 15:22:54.057629 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:22:54.057597    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a4ce9e5_7ac1_4435_84b6_9d6f7e47c76e.slice/crio-892a718d72a126ac3773a9f23bdb7024fd015a57d160c582ef2441a0dd593e73 WatchSource:0}: Error finding container 892a718d72a126ac3773a9f23bdb7024fd015a57d160c582ef2441a0dd593e73: Status 404 returned error can't find the container with id 892a718d72a126ac3773a9f23bdb7024fd015a57d160c582ef2441a0dd593e73
Jul 30 15:22:54.793414 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:54.793363    2582 generic.go:356] "Generic (PLEG): container finished" podID="5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e" containerID="b76662f4dd466cd3f3ebe54c928762d11830ae76f13a85c1dd6ecdd98428e0b9" exitCode=0
Jul 30 15:22:54.793762 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:54.793453    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl" event={"ID":"5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e","Type":"ContainerDied","Data":"b76662f4dd466cd3f3ebe54c928762d11830ae76f13a85c1dd6ecdd98428e0b9"}
Jul 30 15:22:54.793762 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:54.793530    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl" event={"ID":"5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e","Type":"ContainerStarted","Data":"892a718d72a126ac3773a9f23bdb7024fd015a57d160c582ef2441a0dd593e73"}
Jul 30 15:22:55.923544 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:55.923522    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"
Jul 30 15:22:56.085534 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:56.085446    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e-maas-api-service-ca\") pod \"5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e\" (UID: \"5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e\") "
Jul 30 15:22:56.085792 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:56.085770    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e" (UID: "5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:22:56.186503 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:56.186466    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:22:56.803151 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:56.803120    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"
Jul 30 15:22:56.806934 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:56.806908    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl" event={"ID":"5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e","Type":"ContainerDied","Data":"892a718d72a126ac3773a9f23bdb7024fd015a57d160c582ef2441a0dd593e73"}
Jul 30 15:22:56.807058 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:56.806946    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="892a718d72a126ac3773a9f23bdb7024fd015a57d160c582ef2441a0dd593e73"
Jul 30 15:22:59.440569 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:59.440529    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:59.539070 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:59.539044    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:22:59.600308 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:59.600274    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"]
Jul 30 15:22:59.603638 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:59.603607    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73521211-rqjhl"]
Jul 30 15:22:59.604285 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:22:59.604263    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:00.808379 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:00.808343    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e" path="/var/lib/kubelet/pods/5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e/volumes"
Jul 30 15:23:00.842243 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:00.842213    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:01.100636 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:01.100564    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:01.349577 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:01.349543    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:01.675103 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:01.675073    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:02.113284 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:02.113243    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:03.422366 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:03.422324    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:23:03.819082 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:03.819038    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:04.206908 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:04.206876    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:05.114084 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:05.114049    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:05.345889 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:05.345850    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:05.449904 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:05.449878    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:05.544260 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:05.544226    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:14.018640 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:14.018605    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:23:28.783794 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.783755    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"]
Jul 30 15:23:28.784266 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.784177    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e" containerName="revoke-keys"
Jul 30 15:23:28.784266 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.784196    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e" containerName="revoke-keys"
Jul 30 15:23:28.784385 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.784277    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a4ce9e5-7ac1-4435-84b6-9d6f7e47c76e" containerName="revoke-keys"
Jul 30 15:23:28.787240 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.787218    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"
Jul 30 15:23:28.789304 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.789284    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:23:28.789432 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.789282    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 15:23:28.795576 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.795553    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"]
Jul 30 15:23:28.825198 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.825173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e08130-4c15-4ba7-8314-048e740bf923-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs\" (UID: \"e6e08130-4c15-4ba7-8314-048e740bf923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"
Jul 30 15:23:28.926306 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.926272    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e08130-4c15-4ba7-8314-048e740bf923-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs\" (UID: \"e6e08130-4c15-4ba7-8314-048e740bf923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"
Jul 30 15:23:28.927221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:28.927198    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e08130-4c15-4ba7-8314-048e740bf923-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs\" (UID: \"e6e08130-4c15-4ba7-8314-048e740bf923\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"
Jul 30 15:23:29.098321 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:29.098231    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"
Jul 30 15:23:29.225235 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:29.225190    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"]
Jul 30 15:23:29.227968 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:23:29.227940    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e08130_4c15_4ba7_8314_048e740bf923.slice/crio-ca926f18d0728ff734ec327d72ec79be7ca3f3773640c51dab012d8528e3d807 WatchSource:0}: Error finding container ca926f18d0728ff734ec327d72ec79be7ca3f3773640c51dab012d8528e3d807: Status 404 returned error can't find the container with id ca926f18d0728ff734ec327d72ec79be7ca3f3773640c51dab012d8528e3d807
Jul 30 15:23:29.935665 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:29.935625    2582 generic.go:356] "Generic (PLEG): container finished" podID="e6e08130-4c15-4ba7-8314-048e740bf923" containerID="ee41b07768d3b136f39ceb04c0b5ad1982a99ba983e98a5e0819471c2dea58e7" exitCode=0
Jul 30 15:23:29.936046 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:29.935702    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs" event={"ID":"e6e08130-4c15-4ba7-8314-048e740bf923","Type":"ContainerDied","Data":"ee41b07768d3b136f39ceb04c0b5ad1982a99ba983e98a5e0819471c2dea58e7"}
Jul 30 15:23:29.936046 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:29.935758    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs" event={"ID":"e6e08130-4c15-4ba7-8314-048e740bf923","Type":"ContainerStarted","Data":"ca926f18d0728ff734ec327d72ec79be7ca3f3773640c51dab012d8528e3d807"}
Jul 30 15:23:31.067737 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:31.067712    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"
Jul 30 15:23:31.144242 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:31.144209    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e08130-4c15-4ba7-8314-048e740bf923-maas-api-service-ca\") pod \"e6e08130-4c15-4ba7-8314-048e740bf923\" (UID: \"e6e08130-4c15-4ba7-8314-048e740bf923\") "
Jul 30 15:23:31.144625 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:31.144602    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6e08130-4c15-4ba7-8314-048e740bf923-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6e08130-4c15-4ba7-8314-048e740bf923" (UID: "e6e08130-4c15-4ba7-8314-048e740bf923"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:23:31.244885 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:31.244785    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e08130-4c15-4ba7-8314-048e740bf923-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:23:31.944916 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:31.944886    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"
Jul 30 15:23:31.944916 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:31.944911    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs" event={"ID":"e6e08130-4c15-4ba7-8314-048e740bf923","Type":"ContainerDied","Data":"ca926f18d0728ff734ec327d72ec79be7ca3f3773640c51dab012d8528e3d807"}
Jul 30 15:23:31.945127 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:31.944953    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca926f18d0728ff734ec327d72ec79be7ca3f3773640c51dab012d8528e3d807"
Jul 30 15:23:35.514726 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:35.514684    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:35.609111 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:35.609076    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:35.678779 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:35.678744    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"]
Jul 30 15:23:35.681234 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:35.681211    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7ba3a489-wv2xs"]
Jul 30 15:23:35.710319 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:35.710285    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:35.901358 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:35.901280    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:36.117080 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:36.117049    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:36.634724 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:36.634691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:36.763579 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:36.763541    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:36.806766 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:36.806738    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6e08130-4c15-4ba7-8314-048e740bf923" path="/var/lib/kubelet/pods/e6e08130-4c15-4ba7-8314-048e740bf923/volumes"
Jul 30 15:23:36.872199 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:36.872176    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:23:39.329628 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:39.329594    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:23:58.930540 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:23:58.930508    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:24:03.420536 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:03.420500    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:24:05.104190 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:05.104152    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:24:05.211460 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:05.211427    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:24:05.304696 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:05.304671    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:24:05.407343 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:05.407276    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:24:05.884457 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:05.884351    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:24:31.924170 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:31.924132    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:24:38.776737 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:38.776706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:24:39.581887 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:39.581852    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:24:39.650749 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:39.650719    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:24:54.983371 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:24:54.983334    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:01.019328 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:01.019285    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:01.096076 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:01.096045    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:01.197784 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:01.197750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:01.290649 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:01.290562    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:11.825483 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:11.825446    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:22.905972 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:22.905940    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:30.088538 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:30.088499    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:30.154269 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:30.154244    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:30.247988 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:30.247958    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:30.347541 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:30.347472    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:30.449209 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:30.449175    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:31.109812 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:31.109780    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:31.209988 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:31.209956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:31.315139 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:31.315104    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:32.923088 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:32.923056    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:33.126099 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:33.126055    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:25:38.986514 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:38.986441    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:39.088887 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:39.088857    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:39.713494 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:39.713463    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:45.738173 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:45.738135    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:45.838300 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:45.838272    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:46.063728 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:46.063648    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:46.080373 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:46.080345    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:25:46.343342 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:46.343276    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:48.297314 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:48.297279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:49.353500 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:49.353467    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:49.457421 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:49.457375    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:50.476445 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:50.476373    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:50.840721 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:50.840631    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:50.981861 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:50.981831    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:51.084444 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:51.084412    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:25:59.526725 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:59.526691    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:25:59.535521 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:25:59.535489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:14.427364 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:14.427331    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:15.102349 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:15.102316    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:15.206496 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:15.206466    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:15.587130 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:15.587097    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:21.582181 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:21.582139    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:21.644833 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:21.644802    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:21.973726 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:21.973694    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"]
Jul 30 15:26:21.973974 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:21.973962    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6e08130-4c15-4ba7-8314-048e740bf923" containerName="revoke-keys"
Jul 30 15:26:21.974016 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:21.973976    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6e08130-4c15-4ba7-8314-048e740bf923" containerName="revoke-keys"
Jul 30 15:26:21.974050 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:21.974036    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6e08130-4c15-4ba7-8314-048e740bf923" containerName="revoke-keys"
Jul 30 15:26:21.976792 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:21.976777    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"
Jul 30 15:26:21.978953 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:21.978929    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:26:21.979058 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:21.978957    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 15:26:21.985523 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:21.985501    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"]
Jul 30 15:26:22.024116 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:22.024090    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f99f181-f265-4024-8318-2d36b458a0fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x\" (UID: \"1f99f181-f265-4024-8318-2d36b458a0fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"
Jul 30 15:26:22.124646 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:22.124616    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f99f181-f265-4024-8318-2d36b458a0fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x\" (UID: \"1f99f181-f265-4024-8318-2d36b458a0fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"
Jul 30 15:26:22.125189 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:22.125169    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f99f181-f265-4024-8318-2d36b458a0fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x\" (UID: \"1f99f181-f265-4024-8318-2d36b458a0fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"
Jul 30 15:26:22.288012 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:22.287937    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"
Jul 30 15:26:22.411308 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:22.411200    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"]
Jul 30 15:26:22.414072 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:26:22.414035    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f99f181_f265_4024_8318_2d36b458a0fd.slice/crio-43d2f53595d482fabcf53f95b2cd3ca394d1c33072f959e8628ffb36d8a1635e WatchSource:0}: Error finding container 43d2f53595d482fabcf53f95b2cd3ca394d1c33072f959e8628ffb36d8a1635e: Status 404 returned error can't find the container with id 43d2f53595d482fabcf53f95b2cd3ca394d1c33072f959e8628ffb36d8a1635e
Jul 30 15:26:22.647231 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:22.647156    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x" event={"ID":"1f99f181-f265-4024-8318-2d36b458a0fd","Type":"ContainerStarted","Data":"43d2f53595d482fabcf53f95b2cd3ca394d1c33072f959e8628ffb36d8a1635e"}
Jul 30 15:26:23.652583 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:23.652544    2582 generic.go:356] "Generic (PLEG): container finished" podID="1f99f181-f265-4024-8318-2d36b458a0fd" containerID="cc8a01fd541831c447513dd98c0a8ac7551cfffef3918bc302e6794cdc016b04" exitCode=0
Jul 30 15:26:23.652972 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:23.652604    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x" event={"ID":"1f99f181-f265-4024-8318-2d36b458a0fd","Type":"ContainerDied","Data":"cc8a01fd541831c447513dd98c0a8ac7551cfffef3918bc302e6794cdc016b04"}
Jul 30 15:26:24.777916 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:24.777893    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"
Jul 30 15:26:24.849328 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:24.849298    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f99f181-f265-4024-8318-2d36b458a0fd-maas-api-service-ca\") pod \"1f99f181-f265-4024-8318-2d36b458a0fd\" (UID: \"1f99f181-f265-4024-8318-2d36b458a0fd\") "
Jul 30 15:26:24.849646 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:24.849622    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f99f181-f265-4024-8318-2d36b458a0fd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f99f181-f265-4024-8318-2d36b458a0fd" (UID: "1f99f181-f265-4024-8318-2d36b458a0fd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:26:24.950349 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:24.950319    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f99f181-f265-4024-8318-2d36b458a0fd-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:26:25.661696 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:25.661658    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x" event={"ID":"1f99f181-f265-4024-8318-2d36b458a0fd","Type":"ContainerDied","Data":"43d2f53595d482fabcf53f95b2cd3ca394d1c33072f959e8628ffb36d8a1635e"}
Jul 30 15:26:25.661696 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:25.661695    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"
Jul 30 15:26:25.661904 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:25.661713    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43d2f53595d482fabcf53f95b2cd3ca394d1c33072f959e8628ffb36d8a1635e"
Jul 30 15:26:32.535046 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:32.534991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:32.611427 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:32.611382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:32.699164 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:32.699129    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"]
Jul 30 15:26:32.701467 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:32.701445    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ea8d0fa-nfv7x"]
Jul 30 15:26:32.703743 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:32.703719    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:32.773421 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:32.773374    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:32.806058 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:32.805997    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f99f181-f265-4024-8318-2d36b458a0fd" path="/var/lib/kubelet/pods/1f99f181-f265-4024-8318-2d36b458a0fd/volumes"
Jul 30 15:26:33.106149 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:33.106067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:35.622885 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:35.622852    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:26:39.773123 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:39.773080    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:39.873327 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:39.873297    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:40.037698 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:40.037629    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:40.293830 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:40.293750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:40.430203 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:40.430172    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:40.527461 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:40.527429    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:40.678254 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:40.678171    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:43.057866 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:43.057834    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:46.398568 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:46.398536    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:46.462253 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:46.462219    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:46.549286 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:46.549259    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:46.651998 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:46.651928    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:46.800606 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:46.800570    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:52.723116 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:52.723078    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:26:52.783744 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:26:52.783714    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:27:16.123341 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:27:16.123259    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:28:07.288461 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:28:07.288424    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:28:14.857184 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:28:14.857154    2582 scope.go:117] "RemoveContainer" containerID="68a3387c0609127a14ddb3a38c0a9bc085e87065a788945f1d9efc2e36fa3d20"
Jul 30 15:28:37.839114 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:28:37.839038    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:28:55.714788 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:28:55.714756    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:28:56.027640 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:28:56.027554    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:28:56.106119 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:28:56.106082    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:28:59.144052 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:28:59.144022    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:28:59.228500 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:28:59.228472    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:28:59.232008 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:28:59.231986    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:29:00.275722 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:00.275686    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:29:00.432820 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:00.432789    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:29:00.925179 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:00.925147    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:29:01.694368 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:01.694330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:29:09.220427 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:09.220376    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:29:14.873686 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:14.873655    2582 scope.go:117] "RemoveContainer" containerID="969c4d953e23317801693bbcc58795af40dfc2fbeddbdfd82f7c5d1bc9f65d60"
Jul 30 15:29:14.881262 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:14.881243    2582 scope.go:117] "RemoveContainer" containerID="b76662f4dd466cd3f3ebe54c928762d11830ae76f13a85c1dd6ecdd98428e0b9"
Jul 30 15:29:24.254118 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:24.254085    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:29:40.574618 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:40.574577    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:29:44.722139 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:44.722098    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:29:48.123690 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:29:48.123658    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:30:00.174341 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.174306    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96"]
Jul 30 15:30:00.174744 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.174627    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f99f181-f265-4024-8318-2d36b458a0fd" containerName="revoke-keys"
Jul 30 15:30:00.174744 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.174638    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f99f181-f265-4024-8318-2d36b458a0fd" containerName="revoke-keys"
Jul 30 15:30:00.174744 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.174688    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f99f181-f265-4024-8318-2d36b458a0fd" containerName="revoke-keys"
Jul 30 15:30:00.177327 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.177309    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757090-fw7ln"]
Jul 30 15:30:00.177454 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.177434    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96"
Jul 30 15:30:00.180085 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.180068    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:30:00.180183 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.180166    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757090-fw7ln"
Jul 30 15:30:00.187180 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.187165    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96"
Jul 30 15:30:00.193079 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.193060    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757090-fw7ln"
Jul 30 15:30:00.195988 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.195969    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757090-fw7ln"]
Jul 30 15:30:00.199341 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.198418    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96"]
Jul 30 15:30:00.332577 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.332539    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96"]
Jul 30 15:30:00.335207 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:30:00.335162    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2acf21d_b434_4c85_a54b_fa738708d61c.slice/crio-de6bf6a37d2790238ab36ffaafc83aab6737c5a7b03c219e19ef8424e3a39d7a WatchSource:0}: Error finding container de6bf6a37d2790238ab36ffaafc83aab6737c5a7b03c219e19ef8424e3a39d7a: Status 404 returned error can't find the container with id de6bf6a37d2790238ab36ffaafc83aab6737c5a7b03c219e19ef8424e3a39d7a
Jul 30 15:30:00.355870 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.355845    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757090-fw7ln"]
Jul 30 15:30:00.359344 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:30:00.359317    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd670b79_4237_4090_b43d_76df18264d7d.slice/crio-211ea43100dab126883fbbc24831e2716cad8baa49cdb579a39a772a29f121bd WatchSource:0}: Error finding container 211ea43100dab126883fbbc24831e2716cad8baa49cdb579a39a772a29f121bd: Status 404 returned error can't find the container with id 211ea43100dab126883fbbc24831e2716cad8baa49cdb579a39a772a29f121bd
Jul 30 15:30:00.541166 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.541128    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757090-fw7ln" event={"ID":"dd670b79-4237-4090-b43d-76df18264d7d","Type":"ContainerStarted","Data":"211ea43100dab126883fbbc24831e2716cad8baa49cdb579a39a772a29f121bd"}
Jul 30 15:30:00.542247 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:00.542223    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96" event={"ID":"d2acf21d-b434-4c85-a54b-fa738708d61c","Type":"ContainerStarted","Data":"de6bf6a37d2790238ab36ffaafc83aab6737c5a7b03c219e19ef8424e3a39d7a"}
Jul 30 15:30:01.547226 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:01.547142    2582 generic.go:356] "Generic (PLEG): container finished" podID="dd670b79-4237-4090-b43d-76df18264d7d" containerID="73892290d43b160372461db66552ea68523af6b024620f1cc166a3dbddfc8f32" exitCode=0
Jul 30 15:30:01.547633 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:01.547221    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757090-fw7ln" event={"ID":"dd670b79-4237-4090-b43d-76df18264d7d","Type":"ContainerDied","Data":"73892290d43b160372461db66552ea68523af6b024620f1cc166a3dbddfc8f32"}
Jul 30 15:30:01.548477 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:01.548453    2582 generic.go:356] "Generic (PLEG): container finished" podID="d2acf21d-b434-4c85-a54b-fa738708d61c" containerID="9b805c1505df167330d3213fb49074a71010fe341a95744912f7c2753fc6361f" exitCode=0
Jul 30 15:30:01.548574 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:01.548516    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96" event={"ID":"d2acf21d-b434-4c85-a54b-fa738708d61c","Type":"ContainerDied","Data":"9b805c1505df167330d3213fb49074a71010fe341a95744912f7c2753fc6361f"}
Jul 30 15:30:02.677564 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:02.677537    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757090-fw7ln"
Jul 30 15:30:02.710697 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:02.710674    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96"
Jul 30 15:30:03.558805 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:03.558771    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757090-fw7ln"
Jul 30 15:30:03.559001 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:03.558769    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757090-fw7ln" event={"ID":"dd670b79-4237-4090-b43d-76df18264d7d","Type":"ContainerDied","Data":"211ea43100dab126883fbbc24831e2716cad8baa49cdb579a39a772a29f121bd"}
Jul 30 15:30:03.559001 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:03.558933    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="211ea43100dab126883fbbc24831e2716cad8baa49cdb579a39a772a29f121bd"
Jul 30 15:30:03.560126 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:03.560091    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96"
Jul 30 15:30:03.560126 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:03.560102    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96" event={"ID":"d2acf21d-b434-4c85-a54b-fa738708d61c","Type":"ContainerDied","Data":"de6bf6a37d2790238ab36ffaafc83aab6737c5a7b03c219e19ef8424e3a39d7a"}
Jul 30 15:30:03.560323 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:03.560143    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de6bf6a37d2790238ab36ffaafc83aab6737c5a7b03c219e19ef8424e3a39d7a"
Jul 30 15:30:06.622301 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:06.622225    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:30:14.899780 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:14.899745    2582 scope.go:117] "RemoveContainer" containerID="ee41b07768d3b136f39ceb04c0b5ad1982a99ba983e98a5e0819471c2dea58e7"
Jul 30 15:30:19.486266 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.486229    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"]
Jul 30 15:30:19.486719 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.486506    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2acf21d-b434-4c85-a54b-fa738708d61c" containerName="cleanup"
Jul 30 15:30:19.486719 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.486517    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2acf21d-b434-4c85-a54b-fa738708d61c" containerName="cleanup"
Jul 30 15:30:19.486719 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.486537    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd670b79-4237-4090-b43d-76df18264d7d" containerName="cleanup"
Jul 30 15:30:19.486719 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.486543    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd670b79-4237-4090-b43d-76df18264d7d" containerName="cleanup"
Jul 30 15:30:19.486719 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.486588    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2acf21d-b434-4c85-a54b-fa738708d61c" containerName="cleanup"
Jul 30 15:30:19.486719 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.486597    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd670b79-4237-4090-b43d-76df18264d7d" containerName="cleanup"
Jul 30 15:30:19.490964 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.490946    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"
Jul 30 15:30:19.492863 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.492834    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 15:30:19.492989 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.492838    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:30:19.495955 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.495931    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"]
Jul 30 15:30:19.623267 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.623236    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e83ea0c-9991-4903-8511-126b9633efe9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh\" (UID: \"5e83ea0c-9991-4903-8511-126b9633efe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"
Jul 30 15:30:19.724650 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.724623    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e83ea0c-9991-4903-8511-126b9633efe9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh\" (UID: \"5e83ea0c-9991-4903-8511-126b9633efe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"
Jul 30 15:30:19.725183 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.725163    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e83ea0c-9991-4903-8511-126b9633efe9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh\" (UID: \"5e83ea0c-9991-4903-8511-126b9633efe9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"
Jul 30 15:30:19.801848 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.801743    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"
Jul 30 15:30:19.926419 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:19.926366    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"]
Jul 30 15:30:19.929421 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:30:19.929382    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e83ea0c_9991_4903_8511_126b9633efe9.slice/crio-eb002c64bd5ef6d1397192fe31c8280eedde73c0e58fff4c5aa3d0bd0cd468a6 WatchSource:0}: Error finding container eb002c64bd5ef6d1397192fe31c8280eedde73c0e58fff4c5aa3d0bd0cd468a6: Status 404 returned error can't find the container with id eb002c64bd5ef6d1397192fe31c8280eedde73c0e58fff4c5aa3d0bd0cd468a6
Jul 30 15:30:20.633788 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:20.633746    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh" event={"ID":"5e83ea0c-9991-4903-8511-126b9633efe9","Type":"ContainerStarted","Data":"b533bfff55f9b78ac5a5e073e04a576d45fb7909cdc3b7856820009e7e529772"}
Jul 30 15:30:20.634148 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:20.633830    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh" event={"ID":"5e83ea0c-9991-4903-8511-126b9633efe9","Type":"ContainerStarted","Data":"eb002c64bd5ef6d1397192fe31c8280eedde73c0e58fff4c5aa3d0bd0cd468a6"}
Jul 30 15:30:20.649463 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:20.649417    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh" podStartSLOduration=1.649381143 podStartE2EDuration="1.649381143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:30:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:30:20.647177822 +0000 UTC m=+2526.320623935" watchObservedRunningTime="2026-07-30 15:30:20.649381143 +0000 UTC m=+2526.322827271"
Jul 30 15:30:21.639273 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:21.639233    2582 generic.go:356] "Generic (PLEG): container finished" podID="5e83ea0c-9991-4903-8511-126b9633efe9" containerID="b533bfff55f9b78ac5a5e073e04a576d45fb7909cdc3b7856820009e7e529772" exitCode=0
Jul 30 15:30:21.639735 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:21.639289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh" event={"ID":"5e83ea0c-9991-4903-8511-126b9633efe9","Type":"ContainerDied","Data":"b533bfff55f9b78ac5a5e073e04a576d45fb7909cdc3b7856820009e7e529772"}
Jul 30 15:30:22.751098 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:22.751072    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:30:22.766715 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:22.766696    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"
Jul 30 15:30:22.848386 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:22.848361    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e83ea0c-9991-4903-8511-126b9633efe9-maas-api-service-ca\") pod \"5e83ea0c-9991-4903-8511-126b9633efe9\" (UID: \"5e83ea0c-9991-4903-8511-126b9633efe9\") "
Jul 30 15:30:22.848704 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:22.848684    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e83ea0c-9991-4903-8511-126b9633efe9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e83ea0c-9991-4903-8511-126b9633efe9" (UID: "5e83ea0c-9991-4903-8511-126b9633efe9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:30:22.949544 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:22.949521    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e83ea0c-9991-4903-8511-126b9633efe9-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:30:23.649762 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:23.649736    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"
Jul 30 15:30:23.649762 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:23.649747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh" event={"ID":"5e83ea0c-9991-4903-8511-126b9633efe9","Type":"ContainerDied","Data":"eb002c64bd5ef6d1397192fe31c8280eedde73c0e58fff4c5aa3d0bd0cd468a6"}
Jul 30 15:30:23.650027 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:23.649791    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb002c64bd5ef6d1397192fe31c8280eedde73c0e58fff4c5aa3d0bd0cd468a6"
Jul 30 15:30:25.673148 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:25.673114    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:30:33.523630 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:33.523596    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:30:34.982841 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:34.982804    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96"]
Jul 30 15:30:34.987872 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:34.987847    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-2fa717fb-29757090-fkq96"]
Jul 30 15:30:34.991136 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:34.991110    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:30:35.091161 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:35.091132    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:30:35.173455 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:35.173416    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"]
Jul 30 15:30:35.175000 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:35.174975    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2fa717fb-7nxhh"]
Jul 30 15:30:35.176640 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:35.176615    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:30:35.245897 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:35.245817    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:30:36.806514 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:36.806479    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e83ea0c-9991-4903-8511-126b9633efe9" path="/var/lib/kubelet/pods/5e83ea0c-9991-4903-8511-126b9633efe9/volumes"
Jul 30 15:30:36.806850 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:36.806762    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2acf21d-b434-4c85-a54b-fa738708d61c" path="/var/lib/kubelet/pods/d2acf21d-b434-4c85-a54b-fa738708d61c/volumes"
Jul 30 15:30:38.830494 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:38.830464    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:30:41.618248 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:41.618214    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:30:43.863506 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:30:43.863473    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:31:20.514824 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:31:20.514792    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:31:21.053120 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:31:21.053089    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:31:26.728319 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:31:26.728280    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:31:41.273332 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:31:41.273245    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:31:41.571385 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:31:41.571300    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:31:41.925363 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:31:41.925290    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:31:42.026365 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:31:42.026333    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:31:44.625775 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:31:44.625742    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:31:47.672143 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:31:47.672114    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:03.467571 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:03.467539    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:03.571732 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:03.571704    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:03.665166 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:03.665136    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:04.370150 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:04.370104    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:04.467500 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:04.467457    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:05.307300 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:05.307268    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:05.929490 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:05.929455    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:08.531615 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:08.531581    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:32:11.979134 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:11.979102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:12.036077 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:12.036042    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:13.534573 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:13.534539    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:18.229526 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:18.229490    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:32:18.746918 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:18.746888    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:18.859942 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:18.859914    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:18.947486 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:18.947456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:29.499064 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:29.499031    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:29.757480 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:29.757378    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:50.796683 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:50.796649    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:57.314345 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:57.314312    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:57.421550 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:57.421523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:57.685058 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:57.685031    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:58.042076 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:58.041993    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:32:58.336794 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:32:58.336711    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:01.302379 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:01.302347    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:01.401786 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:01.401752    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:02.555536 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:02.555506    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:02.826781 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:02.826697    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:03.010353 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:03.010316    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:03.099009 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:03.098921    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:07.620317 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:07.620235    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:33:14.929408 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:14.929375    2582 scope.go:117] "RemoveContainer" containerID="cc8a01fd541831c447513dd98c0a8ac7551cfffef3918bc302e6794cdc016b04"
Jul 30 15:33:26.396592 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:26.396557    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:26.733411 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:26.733365    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:27.065120 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:27.065037    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:27.734672 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:27.734642    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:30.469675 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:30.469640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:30.578947 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:30.578917    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:31.701595 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.701567    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"]
Jul 30 15:33:31.701956 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.701835    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e83ea0c-9991-4903-8511-126b9633efe9" containerName="revoke-keys"
Jul 30 15:33:31.701956 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.701846    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e83ea0c-9991-4903-8511-126b9633efe9" containerName="revoke-keys"
Jul 30 15:33:31.701956 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.701893    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e83ea0c-9991-4903-8511-126b9633efe9" containerName="revoke-keys"
Jul 30 15:33:31.704868 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.704848    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:31.707185 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.707160    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 15:33:31.707288 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.707206    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q7854\""
Jul 30 15:33:31.707713 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.707699    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 15:33:31.707762 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.707745    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a52c4d-serving-cert\""
Jul 30 15:33:31.713647 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.713625    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"]
Jul 30 15:33:31.727716 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.727688    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:31.805779 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.805755    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb5x2\" (UniqueName: \"kubernetes.io/projected/acb28ea5-04a7-4551-9fd8-7dc36607aca7-kube-api-access-wb5x2\") pod \"maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:31.805907 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.805785    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb28ea5-04a7-4551-9fd8-7dc36607aca7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:31.805907 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.805805    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb28ea5-04a7-4551-9fd8-7dc36607aca7-maas-api-tls\") pod \"maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:31.906946 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.906919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb28ea5-04a7-4551-9fd8-7dc36607aca7-maas-api-tls\") pod \"maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:31.907057 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.906986    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb5x2\" (UniqueName: \"kubernetes.io/projected/acb28ea5-04a7-4551-9fd8-7dc36607aca7-kube-api-access-wb5x2\") pod \"maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:31.907057 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.907010    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb28ea5-04a7-4551-9fd8-7dc36607aca7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:31.907137 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:33:31.907058    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-serving-cert: secret "maas-api-e2e-inf-b-a52c4d-serving-cert" not found
Jul 30 15:33:31.907137 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:33:31.907118    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acb28ea5-04a7-4551-9fd8-7dc36607aca7-maas-api-tls podName:acb28ea5-04a7-4551-9fd8-7dc36607aca7 nodeName:}" failed. No retries permitted until 2026-07-30 15:33:32.407102421 +0000 UTC m=+2718.080548515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/acb28ea5-04a7-4551-9fd8-7dc36607aca7-maas-api-tls") pod "maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl" (UID: "acb28ea5-04a7-4551-9fd8-7dc36607aca7") : secret "maas-api-e2e-inf-b-a52c4d-serving-cert" not found
Jul 30 15:33:31.907647 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.907632    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb28ea5-04a7-4551-9fd8-7dc36607aca7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:31.917107 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:31.917087    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb5x2\" (UniqueName: \"kubernetes.io/projected/acb28ea5-04a7-4551-9fd8-7dc36607aca7-kube-api-access-wb5x2\") pod \"maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:32.274754 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:32.274697    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:32.371873 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:32.371843    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:32.410359 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:32.410329    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb28ea5-04a7-4551-9fd8-7dc36607aca7-maas-api-tls\") pod \"maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:32.412862 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:32.412832    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb28ea5-04a7-4551-9fd8-7dc36607aca7-maas-api-tls\") pod \"maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:32.444891 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:32.444862    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:32.615808 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:32.615728    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:32.742273 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:32.742248    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"]
Jul 30 15:33:32.745231 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:33:32.745202    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb28ea5_04a7_4551_9fd8_7dc36607aca7.slice/crio-c5ca77b79e525a2eb683de9a99b26d58fde7489998a8b1c7441bebee0d88b45f WatchSource:0}: Error finding container c5ca77b79e525a2eb683de9a99b26d58fde7489998a8b1c7441bebee0d88b45f: Status 404 returned error can't find the container with id c5ca77b79e525a2eb683de9a99b26d58fde7489998a8b1c7441bebee0d88b45f
Jul 30 15:33:32.746992 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:32.746978    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 15:33:33.417334 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:33.417293    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl" event={"ID":"acb28ea5-04a7-4551-9fd8-7dc36607aca7","Type":"ContainerStarted","Data":"c5ca77b79e525a2eb683de9a99b26d58fde7489998a8b1c7441bebee0d88b45f"}
Jul 30 15:33:35.427891 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:35.427804    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl" event={"ID":"acb28ea5-04a7-4551-9fd8-7dc36607aca7","Type":"ContainerStarted","Data":"1e93a7459abdd6e5f614ecbbae4235522c900bf624046c57a15aed42c0407f65"}
Jul 30 15:33:35.428237 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:35.427895    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:35.445923 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:35.445877    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl" podStartSLOduration=2.177970758 podStartE2EDuration="4.445860556s" totalImagesPullingTime="2.267889798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:33:35.444620577 +0000 UTC m=+2721.118066692" watchObservedRunningTime="2026-07-30 15:33:35.445860556 +0000 UTC m=+2721.119306704"
Jul 30 15:33:37.925226 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:37.925188    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:33:41.437217 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:41.437185    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:33:53.489753 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:53.489719    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:33:55.499951 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:33:55.499916    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:34:14.263865 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:14.263831    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:34:16.730909 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:16.730870    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:34:28.622982 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:28.622945    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:34:43.129909 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.129836    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"]
Jul 30 15:34:43.133044 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.133024    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.135025 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.134940    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a52c4d\"/\"openshift-service-ca.crt\""
Jul 30 15:34:43.135492 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.135471    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a52c4d\"/\"kube-root-ca.crt\""
Jul 30 15:34:43.135598 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.135491    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a52c4d\"/\"default-dockercfg-jlrp9\""
Jul 30 15:34:43.135598 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.135529    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a52c4d\"/\"test-model-a52c4d-kserve-self-signed-certs\""
Jul 30 15:34:43.146023 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.146003    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"]
Jul 30 15:34:43.149068 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.149038    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-model-cache\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.149068 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.149066    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-dshm\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.149212 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.149083    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tls-certs\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.149212 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.149126    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tmp-dir\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.149212 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.149154    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx8rw\" (UniqueName: \"kubernetes.io/projected/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-kube-api-access-hx8rw\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.149212 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.149168    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-home\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.249529 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.249496    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tmp-dir\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.249679 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.249547    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx8rw\" (UniqueName: \"kubernetes.io/projected/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-kube-api-access-hx8rw\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.249730 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.249708    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-home\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.249774 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.249740    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-model-cache\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.249774 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.249758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-dshm\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.249871 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.249774    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tls-certs\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.249957 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.249939    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tmp-dir\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.250104 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.250088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-home\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.250180 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.250158    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-model-cache\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.252084 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.252061    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-dshm\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.252351 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.252335    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tls-certs\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.256820 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.256796    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx8rw\" (UniqueName: \"kubernetes.io/projected/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-kube-api-access-hx8rw\") pod \"test-model-a52c4d-kserve-74b776f8dc-mc6pm\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") " pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.305347 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.305323    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:34:43.442846 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.442821    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:43.570484 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.570456    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"]
Jul 30 15:34:43.573903 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:34:43.573874    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaaaf982a_255b_47cc_8e91_aa7841fcc1b2.slice/crio-1fd80f4a90e6438c18891aaeb993d1808c6807a5764d825287e0a6ea6251f3cb WatchSource:0}: Error finding container 1fd80f4a90e6438c18891aaeb993d1808c6807a5764d825287e0a6ea6251f3cb: Status 404 returned error can't find the container with id 1fd80f4a90e6438c18891aaeb993d1808c6807a5764d825287e0a6ea6251f3cb
Jul 30 15:34:43.706620 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.706540    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm" event={"ID":"aaaf982a-255b-47cc-8e91-aa7841fcc1b2","Type":"ContainerStarted","Data":"17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97"}
Jul 30 15:34:43.706620 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.706595    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm" event={"ID":"aaaf982a-255b-47cc-8e91-aa7841fcc1b2","Type":"ContainerStarted","Data":"1fd80f4a90e6438c18891aaeb993d1808c6807a5764d825287e0a6ea6251f3cb"}
Jul 30 15:34:43.723183 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:43.723139    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm" podStartSLOduration=0.723124111 podStartE2EDuration="723.124111ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:34:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:34:43.721627124 +0000 UTC m=+2789.395073237" watchObservedRunningTime="2026-07-30 15:34:43.723124111 +0000 UTC m=+2789.396570223"
Jul 30 15:34:47.822666 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:47.822634    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:34:53.443781 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:53.443743    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:53.444178 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:53.443802    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:53.456105 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:53.456080    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:34:53.757312 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:34:53.757236    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:35:02.525041 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:02.524998    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:35:35.977671 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:35.977636    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:36.948127 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:36.948092    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"]
Jul 30 15:35:37.085492 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.085463    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"]
Jul 30 15:35:37.088447 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.088430    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"
Jul 30 15:35:37.090419 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.090380    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:35:37.095730 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.095678    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"]
Jul 30 15:35:37.179988 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.179952    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d269efb-1a89-4eba-9aec-ae89e49af6d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx\" (UID: \"6d269efb-1a89-4eba-9aec-ae89e49af6d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"
Jul 30 15:35:37.280761 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.280667    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d269efb-1a89-4eba-9aec-ae89e49af6d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx\" (UID: \"6d269efb-1a89-4eba-9aec-ae89e49af6d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"
Jul 30 15:35:37.281253 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.281235    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d269efb-1a89-4eba-9aec-ae89e49af6d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx\" (UID: \"6d269efb-1a89-4eba-9aec-ae89e49af6d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"
Jul 30 15:35:37.398944 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.398913    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"
Jul 30 15:35:37.530622 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.530590    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"]
Jul 30 15:35:37.533912 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:35:37.533891    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d269efb_1a89_4eba_9aec_ae89e49af6d8.slice/crio-6be1c698b9f136a0d5e3c993388c7907728a844111469adf98b3a6357a9eec86 WatchSource:0}: Error finding container 6be1c698b9f136a0d5e3c993388c7907728a844111469adf98b3a6357a9eec86: Status 404 returned error can't find the container with id 6be1c698b9f136a0d5e3c993388c7907728a844111469adf98b3a6357a9eec86
Jul 30 15:35:37.911070 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.910987    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:37.928013 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.927978    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx" event={"ID":"6d269efb-1a89-4eba-9aec-ae89e49af6d8","Type":"ContainerStarted","Data":"1f2e2c2491313baf3b2cb5a40ba5d2dbe7cb1cccb32550698dd104c3c052d24f"}
Jul 30 15:35:37.928130 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.928028    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx" event={"ID":"6d269efb-1a89-4eba-9aec-ae89e49af6d8","Type":"ContainerStarted","Data":"6be1c698b9f136a0d5e3c993388c7907728a844111469adf98b3a6357a9eec86"}
Jul 30 15:35:37.942416 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:37.942350    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx" podStartSLOduration=0.942330057 podStartE2EDuration="942.330057ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:35:37.94141893 +0000 UTC m=+2843.614865036" watchObservedRunningTime="2026-07-30 15:35:37.942330057 +0000 UTC m=+2843.615776170"
Jul 30 15:35:38.933297 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:38.933261    2582 generic.go:356] "Generic (PLEG): container finished" podID="6d269efb-1a89-4eba-9aec-ae89e49af6d8" containerID="1f2e2c2491313baf3b2cb5a40ba5d2dbe7cb1cccb32550698dd104c3c052d24f" exitCode=0
Jul 30 15:35:38.933693 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:38.933354    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx" event={"ID":"6d269efb-1a89-4eba-9aec-ae89e49af6d8","Type":"ContainerDied","Data":"1f2e2c2491313baf3b2cb5a40ba5d2dbe7cb1cccb32550698dd104c3c052d24f"}
Jul 30 15:35:40.064456 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:40.064434    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"
Jul 30 15:35:40.204613 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:40.204531    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d269efb-1a89-4eba-9aec-ae89e49af6d8-maas-api-service-ca\") pod \"6d269efb-1a89-4eba-9aec-ae89e49af6d8\" (UID: \"6d269efb-1a89-4eba-9aec-ae89e49af6d8\") "
Jul 30 15:35:40.206480 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:40.205306    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d269efb-1a89-4eba-9aec-ae89e49af6d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6d269efb-1a89-4eba-9aec-ae89e49af6d8" (UID: "6d269efb-1a89-4eba-9aec-ae89e49af6d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:35:40.305782 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:40.305754    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d269efb-1a89-4eba-9aec-ae89e49af6d8-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:35:40.942613 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:40.942579    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"
Jul 30 15:35:40.942613 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:40.942599    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx" event={"ID":"6d269efb-1a89-4eba-9aec-ae89e49af6d8","Type":"ContainerDied","Data":"6be1c698b9f136a0d5e3c993388c7907728a844111469adf98b3a6357a9eec86"}
Jul 30 15:35:40.942806 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:40.942641    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6be1c698b9f136a0d5e3c993388c7907728a844111469adf98b3a6357a9eec86"
Jul 30 15:35:41.954938 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:41.954911    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:42.022086 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:42.022055    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:42.122790 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:42.122760    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:42.128808 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:42.128785    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"]
Jul 30 15:35:42.132023 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:42.131989    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f03573-ksnsx"]
Jul 30 15:35:42.224442 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:42.224352    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:42.519455 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:42.519366    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:42.806189 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:42.806117    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d269efb-1a89-4eba-9aec-ae89e49af6d8" path="/var/lib/kubelet/pods/6d269efb-1a89-4eba-9aec-ae89e49af6d8/volumes"
Jul 30 15:35:43.223811 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:43.223777    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:35:47.109109 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:47.109079    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:47.594159 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:47.594122    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:51.968345 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:51.968287    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm" podUID="aaaf982a-255b-47cc-8e91-aa7841fcc1b2" containerName="main" containerID="cri-o://17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97" gracePeriod=45
Jul 30 15:35:52.217480 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.217455    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:35:52.298686 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.298602    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-home\") pod \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") "
Jul 30 15:35:52.298686 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.298657    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-dshm\") pod \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") "
Jul 30 15:35:52.298895 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.298695    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-model-cache\") pod \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") "
Jul 30 15:35:52.298895 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.298740    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tls-certs\") pod \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") "
Jul 30 15:35:52.298895 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.298769    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx8rw\" (UniqueName: \"kubernetes.io/projected/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-kube-api-access-hx8rw\") pod \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") "
Jul 30 15:35:52.298895 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.298800    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tmp-dir\") pod \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\" (UID: \"aaaf982a-255b-47cc-8e91-aa7841fcc1b2\") "
Jul 30 15:35:52.298895 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.298850    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-home" (OuterVolumeSpecName: "home") pod "aaaf982a-255b-47cc-8e91-aa7841fcc1b2" (UID: "aaaf982a-255b-47cc-8e91-aa7841fcc1b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:35:52.299147 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.298943    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-model-cache" (OuterVolumeSpecName: "model-cache") pod "aaaf982a-255b-47cc-8e91-aa7841fcc1b2" (UID: "aaaf982a-255b-47cc-8e91-aa7841fcc1b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:35:52.299147 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.299031    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-home\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:35:52.299147 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.299048    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-model-cache\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:35:52.299147 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.299084    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aaaf982a-255b-47cc-8e91-aa7841fcc1b2" (UID: "aaaf982a-255b-47cc-8e91-aa7841fcc1b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:35:52.301039 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.301015    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-dshm" (OuterVolumeSpecName: "dshm") pod "aaaf982a-255b-47cc-8e91-aa7841fcc1b2" (UID: "aaaf982a-255b-47cc-8e91-aa7841fcc1b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:35:52.301039 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.301023    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aaaf982a-255b-47cc-8e91-aa7841fcc1b2" (UID: "aaaf982a-255b-47cc-8e91-aa7841fcc1b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:35:52.301177 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.301045    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-kube-api-access-hx8rw" (OuterVolumeSpecName: "kube-api-access-hx8rw") pod "aaaf982a-255b-47cc-8e91-aa7841fcc1b2" (UID: "aaaf982a-255b-47cc-8e91-aa7841fcc1b2"). InnerVolumeSpecName "kube-api-access-hx8rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:35:52.399720 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.399690    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tls-certs\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:35:52.399720 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.399717    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx8rw\" (UniqueName: \"kubernetes.io/projected/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-kube-api-access-hx8rw\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:35:52.399720 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.399726    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-tmp-dir\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:35:52.399949 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.399734    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaaf982a-255b-47cc-8e91-aa7841fcc1b2-dshm\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:35:52.422654 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.422627    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:35:52.991570 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.991536    2582 generic.go:356] "Generic (PLEG): container finished" podID="aaaf982a-255b-47cc-8e91-aa7841fcc1b2" containerID="17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97" exitCode=0
Jul 30 15:35:52.992033 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.991577    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"
Jul 30 15:35:52.992033 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.991615    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm" event={"ID":"aaaf982a-255b-47cc-8e91-aa7841fcc1b2","Type":"ContainerDied","Data":"17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97"}
Jul 30 15:35:52.992033 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.991652    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm" event={"ID":"aaaf982a-255b-47cc-8e91-aa7841fcc1b2","Type":"ContainerDied","Data":"1fd80f4a90e6438c18891aaeb993d1808c6807a5764d825287e0a6ea6251f3cb"}
Jul 30 15:35:52.992033 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.991664    2582 scope.go:117] "RemoveContainer" containerID="17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97"
Jul 30 15:35:52.999428 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.999412    2582 scope.go:117] "RemoveContainer" containerID="17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97"
Jul 30 15:35:52.999707 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:35:52.999680    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97\": container with ID starting with 17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97 not found: ID does not exist" containerID="17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97"
Jul 30 15:35:52.999752 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:52.999716    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97"} err="failed to get container status \"17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97\": rpc error: code = NotFound desc = could not find container \"17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97\": container with ID starting with 17bd7cf35d59243734326bf6c0dd8a4550812a7e603ced4d791a44a5e25ddd97 not found: ID does not exist"
Jul 30 15:35:53.005338 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:53.005317    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"]
Jul 30 15:35:53.011512 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:53.011490    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a52c4d/test-model-a52c4d-kserve-74b776f8dc-mc6pm"]
Jul 30 15:35:53.879839 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:53.879803    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:54.174443 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.174342    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:54.424350 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.424302    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:54.541350 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.541307    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"]
Jul 30 15:35:54.541609 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.541597    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d269efb-1a89-4eba-9aec-ae89e49af6d8" containerName="revoke-keys"
Jul 30 15:35:54.541653 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.541611    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d269efb-1a89-4eba-9aec-ae89e49af6d8" containerName="revoke-keys"
Jul 30 15:35:54.541653 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.541633    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaaf982a-255b-47cc-8e91-aa7841fcc1b2" containerName="main"
Jul 30 15:35:54.541653 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.541638    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaaf982a-255b-47cc-8e91-aa7841fcc1b2" containerName="main"
Jul 30 15:35:54.541747 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.541679    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d269efb-1a89-4eba-9aec-ae89e49af6d8" containerName="revoke-keys"
Jul 30 15:35:54.541747 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.541687    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaaf982a-255b-47cc-8e91-aa7841fcc1b2" containerName="main"
Jul 30 15:35:54.545744 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.545728    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"
Jul 30 15:35:54.548367 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.548342    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:35:54.550368 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.550343    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"]
Jul 30 15:35:54.616253 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.616222    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8574949b-6e6d-4604-b95e-21d364b96ee9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5\" (UID: \"8574949b-6e6d-4604-b95e-21d364b96ee9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"
Jul 30 15:35:54.717140 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.717104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8574949b-6e6d-4604-b95e-21d364b96ee9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5\" (UID: \"8574949b-6e6d-4604-b95e-21d364b96ee9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"
Jul 30 15:35:54.717722 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.717702    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8574949b-6e6d-4604-b95e-21d364b96ee9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5\" (UID: \"8574949b-6e6d-4604-b95e-21d364b96ee9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"
Jul 30 15:35:54.808455 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.808374    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaaf982a-255b-47cc-8e91-aa7841fcc1b2" path="/var/lib/kubelet/pods/aaaf982a-255b-47cc-8e91-aa7841fcc1b2/volumes"
Jul 30 15:35:54.858218 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.858182    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"
Jul 30 15:35:54.992578 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:54.992532    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"]
Jul 30 15:35:54.995233 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:35:54.995205    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8574949b_6e6d_4604_b95e_21d364b96ee9.slice/crio-212c5420ffa1f5c1baa2984d4a9984699e3f05a3e9e17a6516d4e4368faad2bd WatchSource:0}: Error finding container 212c5420ffa1f5c1baa2984d4a9984699e3f05a3e9e17a6516d4e4368faad2bd: Status 404 returned error can't find the container with id 212c5420ffa1f5c1baa2984d4a9984699e3f05a3e9e17a6516d4e4368faad2bd
Jul 30 15:35:55.005622 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:55.005593    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5" event={"ID":"8574949b-6e6d-4604-b95e-21d364b96ee9","Type":"ContainerStarted","Data":"212c5420ffa1f5c1baa2984d4a9984699e3f05a3e9e17a6516d4e4368faad2bd"}
Jul 30 15:35:56.012191 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:56.012158    2582 generic.go:356] "Generic (PLEG): container finished" podID="8574949b-6e6d-4604-b95e-21d364b96ee9" containerID="851c6a6270234691a9e5932c12b0e8232e4d1000d863a57452a5f8846105260e" exitCode=0
Jul 30 15:35:56.012564 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:56.012213    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5" event={"ID":"8574949b-6e6d-4604-b95e-21d364b96ee9","Type":"ContainerDied","Data":"851c6a6270234691a9e5932c12b0e8232e4d1000d863a57452a5f8846105260e"}
Jul 30 15:35:56.677095 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:56.677059    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:56.738952 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:56.738927    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:35:57.142503 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:57.142477    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"
Jul 30 15:35:57.239069 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:57.239031    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8574949b-6e6d-4604-b95e-21d364b96ee9-maas-api-service-ca\") pod \"8574949b-6e6d-4604-b95e-21d364b96ee9\" (UID: \"8574949b-6e6d-4604-b95e-21d364b96ee9\") "
Jul 30 15:35:57.239362 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:57.239339    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8574949b-6e6d-4604-b95e-21d364b96ee9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8574949b-6e6d-4604-b95e-21d364b96ee9" (UID: "8574949b-6e6d-4604-b95e-21d364b96ee9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:35:57.340325 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:57.340254    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8574949b-6e6d-4604-b95e-21d364b96ee9-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:35:58.021608 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:58.021577    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5" event={"ID":"8574949b-6e6d-4604-b95e-21d364b96ee9","Type":"ContainerDied","Data":"212c5420ffa1f5c1baa2984d4a9984699e3f05a3e9e17a6516d4e4368faad2bd"}
Jul 30 15:35:58.021608 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:58.021602    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"
Jul 30 15:35:58.021822 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:58.021625    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="212c5420ffa1f5c1baa2984d4a9984699e3f05a3e9e17a6516d4e4368faad2bd"
Jul 30 15:35:59.851593 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:59.851562    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"]
Jul 30 15:35:59.852038 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:59.851764    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl" podUID="acb28ea5-04a7-4551-9fd8-7dc36607aca7" containerName="maas-api" containerID="cri-o://1e93a7459abdd6e5f614ecbbae4235522c900bf624046c57a15aed42c0407f65" gracePeriod=30
Jul 30 15:35:59.875286 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:35:59.875262    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:00.033413 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.033359    2582 generic.go:356] "Generic (PLEG): container finished" podID="acb28ea5-04a7-4551-9fd8-7dc36607aca7" containerID="1e93a7459abdd6e5f614ecbbae4235522c900bf624046c57a15aed42c0407f65" exitCode=0
Jul 30 15:36:00.033570 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.033453    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl" event={"ID":"acb28ea5-04a7-4551-9fd8-7dc36607aca7","Type":"ContainerDied","Data":"1e93a7459abdd6e5f614ecbbae4235522c900bf624046c57a15aed42c0407f65"}
Jul 30 15:36:00.036666 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.036614    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"]
Jul 30 15:36:00.038447 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.038427    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a52c4d-bddg5"]
Jul 30 15:36:00.100777 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.100755    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:36:00.165022 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.164937    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wb5x2\" (UniqueName: \"kubernetes.io/projected/acb28ea5-04a7-4551-9fd8-7dc36607aca7-kube-api-access-wb5x2\") pod \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") "
Jul 30 15:36:00.165022 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.164968    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb28ea5-04a7-4551-9fd8-7dc36607aca7-openshift-service-ca\") pod \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") "
Jul 30 15:36:00.165022 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.164993    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb28ea5-04a7-4551-9fd8-7dc36607aca7-maas-api-tls\") pod \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\" (UID: \"acb28ea5-04a7-4551-9fd8-7dc36607aca7\") "
Jul 30 15:36:00.165423 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.165377    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acb28ea5-04a7-4551-9fd8-7dc36607aca7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "acb28ea5-04a7-4551-9fd8-7dc36607aca7" (UID: "acb28ea5-04a7-4551-9fd8-7dc36607aca7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:36:00.167345 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.167326    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acb28ea5-04a7-4551-9fd8-7dc36607aca7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "acb28ea5-04a7-4551-9fd8-7dc36607aca7" (UID: "acb28ea5-04a7-4551-9fd8-7dc36607aca7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:36:00.167430 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.167326    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acb28ea5-04a7-4551-9fd8-7dc36607aca7-kube-api-access-wb5x2" (OuterVolumeSpecName: "kube-api-access-wb5x2") pod "acb28ea5-04a7-4551-9fd8-7dc36607aca7" (UID: "acb28ea5-04a7-4551-9fd8-7dc36607aca7"). InnerVolumeSpecName "kube-api-access-wb5x2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:36:00.265964 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.265910    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb5x2\" (UniqueName: \"kubernetes.io/projected/acb28ea5-04a7-4551-9fd8-7dc36607aca7-kube-api-access-wb5x2\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:36:00.265964 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.265956    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acb28ea5-04a7-4551-9fd8-7dc36607aca7-openshift-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:36:00.265964 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.265969    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acb28ea5-04a7-4551-9fd8-7dc36607aca7-maas-api-tls\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:36:00.583045 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.583005    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:00.639501 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.639462    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:00.806836 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:00.806805    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8574949b-6e6d-4604-b95e-21d364b96ee9" path="/var/lib/kubelet/pods/8574949b-6e6d-4604-b95e-21d364b96ee9/volumes"
Jul 30 15:36:01.039079 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:01.039046    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"
Jul 30 15:36:01.039545 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:01.039076    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl" event={"ID":"acb28ea5-04a7-4551-9fd8-7dc36607aca7","Type":"ContainerDied","Data":"c5ca77b79e525a2eb683de9a99b26d58fde7489998a8b1c7441bebee0d88b45f"}
Jul 30 15:36:01.039545 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:01.039131    2582 scope.go:117] "RemoveContainer" containerID="1e93a7459abdd6e5f614ecbbae4235522c900bf624046c57a15aed42c0407f65"
Jul 30 15:36:01.054129 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:01.054109    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"]
Jul 30 15:36:01.057606 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:01.057586    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a52c4d-cf5bcb997-kf7nl"]
Jul 30 15:36:02.125090 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:02.125055    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:36:02.810479 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:02.810443    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acb28ea5-04a7-4551-9fd8-7dc36607aca7" path="/var/lib/kubelet/pods/acb28ea5-04a7-4551-9fd8-7dc36607aca7/volumes"
Jul 30 15:36:06.541224 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:06.541140    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:06.939630 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:06.939595    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:07.701247 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:07.701213    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:07.802118 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:07.802082    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:07.908744 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:07.908702    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:10.316116 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:10.316067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:10.431798 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:10.431768    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:11.498330 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.498297    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"]
Jul 30 15:36:11.498708 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.498576    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acb28ea5-04a7-4551-9fd8-7dc36607aca7" containerName="maas-api"
Jul 30 15:36:11.498708 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.498587    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="acb28ea5-04a7-4551-9fd8-7dc36607aca7" containerName="maas-api"
Jul 30 15:36:11.498708 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.498608    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8574949b-6e6d-4604-b95e-21d364b96ee9" containerName="revoke-keys"
Jul 30 15:36:11.498708 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.498613    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8574949b-6e6d-4604-b95e-21d364b96ee9" containerName="revoke-keys"
Jul 30 15:36:11.498708 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.498656    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8574949b-6e6d-4604-b95e-21d364b96ee9" containerName="revoke-keys"
Jul 30 15:36:11.498708 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.498665    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="acb28ea5-04a7-4551-9fd8-7dc36607aca7" containerName="maas-api"
Jul 30 15:36:11.502894 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.502875    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:11.504876 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.504853    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a1c473-serving-cert\""
Jul 30 15:36:11.505005 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.504909    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 15:36:11.505005 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.504943    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q7854\""
Jul 30 15:36:11.505322 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.505306    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 15:36:11.512845 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.512817    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"]
Jul 30 15:36:11.555792 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.555758    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm96p\" (UniqueName: \"kubernetes.io/projected/68691566-6030-4d03-be72-58ddcad859d9-kube-api-access-wm96p\") pod \"maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:11.555892 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.555823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68691566-6030-4d03-be72-58ddcad859d9-maas-api-tls\") pod \"maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:11.555892 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.555854    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68691566-6030-4d03-be72-58ddcad859d9-openshift-service-ca\") pod \"maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:11.590461 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.590435    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:11.656375 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.656343    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm96p\" (UniqueName: \"kubernetes.io/projected/68691566-6030-4d03-be72-58ddcad859d9-kube-api-access-wm96p\") pod \"maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:11.656543 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.656426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68691566-6030-4d03-be72-58ddcad859d9-maas-api-tls\") pod \"maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:11.656543 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.656448    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68691566-6030-4d03-be72-58ddcad859d9-openshift-service-ca\") pod \"maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:11.656622 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:36:11.656564    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-serving-cert: secret "maas-api-e2e-api-a-a1c473-serving-cert" not found
Jul 30 15:36:11.656664 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:36:11.656651    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68691566-6030-4d03-be72-58ddcad859d9-maas-api-tls podName:68691566-6030-4d03-be72-58ddcad859d9 nodeName:}" failed. No retries permitted until 2026-07-30 15:36:12.156628384 +0000 UTC m=+2877.830074479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/68691566-6030-4d03-be72-58ddcad859d9-maas-api-tls") pod "maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz" (UID: "68691566-6030-4d03-be72-58ddcad859d9") : secret "maas-api-e2e-api-a-a1c473-serving-cert" not found
Jul 30 15:36:11.657131 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.657108    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68691566-6030-4d03-be72-58ddcad859d9-openshift-service-ca\") pod \"maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:11.664544 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.664518    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm96p\" (UniqueName: \"kubernetes.io/projected/68691566-6030-4d03-be72-58ddcad859d9-kube-api-access-wm96p\") pod \"maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:11.765574 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:11.765513    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:12.029671 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:12.029594    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:12.102836 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:12.102807    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:12.159119 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:12.159091    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68691566-6030-4d03-be72-58ddcad859d9-maas-api-tls\") pod \"maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:12.161591 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:12.161569    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68691566-6030-4d03-be72-58ddcad859d9-maas-api-tls\") pod \"maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:12.413215 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:12.413138    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:12.543004 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:12.542970    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"]
Jul 30 15:36:12.546366 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:36:12.546342    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68691566_6030_4d03_be72_58ddcad859d9.slice/crio-9c00dcf7af497a199a1d9150669edba3a382574d07d86ce6b4a09121c5527633 WatchSource:0}: Error finding container 9c00dcf7af497a199a1d9150669edba3a382574d07d86ce6b4a09121c5527633: Status 404 returned error can't find the container with id 9c00dcf7af497a199a1d9150669edba3a382574d07d86ce6b4a09121c5527633
Jul 30 15:36:13.089895 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:13.089861    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz" event={"ID":"68691566-6030-4d03-be72-58ddcad859d9","Type":"ContainerStarted","Data":"9c00dcf7af497a199a1d9150669edba3a382574d07d86ce6b4a09121c5527633"}
Jul 30 15:36:14.095752 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:14.095668    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz" event={"ID":"68691566-6030-4d03-be72-58ddcad859d9","Type":"ContainerStarted","Data":"92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84"}
Jul 30 15:36:14.095752 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:14.095743    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:14.113138 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:14.113090    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz" podStartSLOduration=1.876038988 podStartE2EDuration="3.113072775s" totalImagesPullingTime="1.237033787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:36:14.111188214 +0000 UTC m=+2879.784634326" watchObservedRunningTime="2026-07-30 15:36:14.113072775 +0000 UTC m=+2879.786518866"
Jul 30 15:36:14.957435 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:14.957387    2582 scope.go:117] "RemoveContainer" containerID="9b805c1505df167330d3213fb49074a71010fe341a95744912f7c2753fc6361f"
Jul 30 15:36:16.023797 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:16.023755    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:36:20.105036 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:20.105005    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:36:35.880572 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:35.880525    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:35.939132 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:35.939094    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:36.251097 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:36.251061    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:36.484785 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:36.484748    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:39.493506 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:39.493471    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:39.585846 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:39.585815    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:40.586631 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.586592    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"]
Jul 30 15:36:40.589987 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.589968    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:40.592026 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.592007    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e2257c-serving-cert\""
Jul 30 15:36:40.600870 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.600844    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"]
Jul 30 15:36:40.632114 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.632086    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:40.676881 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.676856    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jb4k\" (UniqueName: \"kubernetes.io/projected/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-kube-api-access-2jb4k\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:40.676984 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.676913    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-openshift-service-ca\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:40.677034 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.676985    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:40.778209 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.778180    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:40.778328 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.778224    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jb4k\" (UniqueName: \"kubernetes.io/projected/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-kube-api-access-2jb4k\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:40.778328 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.778267    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-openshift-service-ca\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:40.778328 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:36:40.778317    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-serving-cert: secret "maas-api-e2e-api-b-e2257c-serving-cert" not found
Jul 30 15:36:40.778470 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:36:40.778409    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls podName:79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0 nodeName:}" failed. No retries permitted until 2026-07-30 15:36:41.278373997 +0000 UTC m=+2906.951820088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls") pod "maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk" (UID: "79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0") : secret "maas-api-e2e-api-b-e2257c-serving-cert" not found
Jul 30 15:36:40.778867 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.778849    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-openshift-service-ca\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:40.788109 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:40.788089    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jb4k\" (UniqueName: \"kubernetes.io/projected/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-kube-api-access-2jb4k\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:41.107021 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:41.106991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:41.207483 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:41.207442    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:41.282015 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:41.281981    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:41.282174 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:36:41.282113    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-serving-cert: secret "maas-api-e2e-api-b-e2257c-serving-cert" not found
Jul 30 15:36:41.282218 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:36:41.282175    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls podName:79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0 nodeName:}" failed. No retries permitted until 2026-07-30 15:36:42.282160794 +0000 UTC m=+2907.955606888 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls") pod "maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk" (UID: "79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0") : secret "maas-api-e2e-api-b-e2257c-serving-cert" not found
Jul 30 15:36:41.914276 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:41.914248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:36:42.291423 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:42.291375    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:42.293845 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:42.293825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls\") pod \"maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:42.400311 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:42.400274    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:42.527312 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:42.527290    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"]
Jul 30 15:36:42.529903 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:36:42.529875    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ea5f9b_7ae2_4a10_b78c_b5a7779fc8b0.slice/crio-bce32e2487201de36364022234563b97a7592860b60c16a1d38401cde8e66451 WatchSource:0}: Error finding container bce32e2487201de36364022234563b97a7592860b60c16a1d38401cde8e66451: Status 404 returned error can't find the container with id bce32e2487201de36364022234563b97a7592860b60c16a1d38401cde8e66451
Jul 30 15:36:43.215890 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:43.215854    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk" event={"ID":"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0","Type":"ContainerStarted","Data":"c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf"}
Jul 30 15:36:43.216301 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:43.215908    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk" event={"ID":"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0","Type":"ContainerStarted","Data":"bce32e2487201de36364022234563b97a7592860b60c16a1d38401cde8e66451"}
Jul 30 15:36:43.216301 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:43.216001    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:36:43.232541 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:43.232501    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk" podStartSLOduration=2.876039517 podStartE2EDuration="3.232483528s" totalImagesPullingTime="356.444011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:36:43.231302899 +0000 UTC m=+2908.904749012" watchObservedRunningTime="2026-07-30 15:36:43.232483528 +0000 UTC m=+2908.905929646"
Jul 30 15:36:46.923082 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:46.923046    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:36:49.225413 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:36:49.225361    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:37:04.599407 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:04.599348    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:08.033143 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:08.033111    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"]
Jul 30 15:37:08.036085 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:08.036071    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"
Jul 30 15:37:08.039471 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:08.039448    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:37:08.052760 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:08.052740    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"]
Jul 30 15:37:08.074262 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:08.074237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c9be75-30e4-4f81-91a3-8a032fd5912f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a1c473-ktggv\" (UID: \"64c9be75-30e4-4f81-91a3-8a032fd5912f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"
Jul 30 15:37:08.175433 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:08.175382    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c9be75-30e4-4f81-91a3-8a032fd5912f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a1c473-ktggv\" (UID: \"64c9be75-30e4-4f81-91a3-8a032fd5912f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"
Jul 30 15:37:08.176019 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:08.175992    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c9be75-30e4-4f81-91a3-8a032fd5912f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a1c473-ktggv\" (UID: \"64c9be75-30e4-4f81-91a3-8a032fd5912f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"
Jul 30 15:37:08.346051 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:08.345954    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"
Jul 30 15:37:08.489237 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:08.489200    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"]
Jul 30 15:37:08.492156 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:37:08.492129    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c9be75_30e4_4f81_91a3_8a032fd5912f.slice/crio-f9b102c710a0fb202cecdf7e922929651440d54e6c7e52db1ab0726b13b4c7c8 WatchSource:0}: Error finding container f9b102c710a0fb202cecdf7e922929651440d54e6c7e52db1ab0726b13b4c7c8: Status 404 returned error can't find the container with id f9b102c710a0fb202cecdf7e922929651440d54e6c7e52db1ab0726b13b4c7c8
Jul 30 15:37:09.324008 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:09.323968    2582 generic.go:356] "Generic (PLEG): container finished" podID="64c9be75-30e4-4f81-91a3-8a032fd5912f" containerID="0e5c9db0139c5d4a94f7d0329ef8e7c03cae6e09b27c6ad232906554b9db9fbf" exitCode=0
Jul 30 15:37:09.324376 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:09.324014    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv" event={"ID":"64c9be75-30e4-4f81-91a3-8a032fd5912f","Type":"ContainerDied","Data":"0e5c9db0139c5d4a94f7d0329ef8e7c03cae6e09b27c6ad232906554b9db9fbf"}
Jul 30 15:37:09.324376 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:09.324067    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv" event={"ID":"64c9be75-30e4-4f81-91a3-8a032fd5912f","Type":"ContainerStarted","Data":"f9b102c710a0fb202cecdf7e922929651440d54e6c7e52db1ab0726b13b4c7c8"}
Jul 30 15:37:10.455870 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:10.455843    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"
Jul 30 15:37:10.489120 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:10.489097    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c9be75-30e4-4f81-91a3-8a032fd5912f-maas-api-service-ca\") pod \"64c9be75-30e4-4f81-91a3-8a032fd5912f\" (UID: \"64c9be75-30e4-4f81-91a3-8a032fd5912f\") "
Jul 30 15:37:10.489434 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:10.489413    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c9be75-30e4-4f81-91a3-8a032fd5912f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "64c9be75-30e4-4f81-91a3-8a032fd5912f" (UID: "64c9be75-30e4-4f81-91a3-8a032fd5912f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:37:10.589739 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:10.589659    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c9be75-30e4-4f81-91a3-8a032fd5912f-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:37:11.333145 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:11.333115    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"
Jul 30 15:37:11.333372 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:11.333161    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv" event={"ID":"64c9be75-30e4-4f81-91a3-8a032fd5912f","Type":"ContainerDied","Data":"f9b102c710a0fb202cecdf7e922929651440d54e6c7e52db1ab0726b13b4c7c8"}
Jul 30 15:37:11.333372 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:11.333210    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9b102c710a0fb202cecdf7e922929651440d54e6c7e52db1ab0726b13b4c7c8"
Jul 30 15:37:14.980556 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:14.980527    2582 scope.go:117] "RemoveContainer" containerID="b533bfff55f9b78ac5a5e073e04a576d45fb7909cdc3b7856820009e7e529772"
Jul 30 15:37:23.534258 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.534223    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"]
Jul 30 15:37:23.534661 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.534476    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz" podUID="68691566-6030-4d03-be72-58ddcad859d9" containerName="maas-api" containerID="cri-o://92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84" gracePeriod=30
Jul 30 15:37:23.563062 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.563033    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:23.625477 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.625444    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:23.724258 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.724228    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"]
Jul 30 15:37:23.726509 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.726481    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:23.726625 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.726594    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a1c473-ktggv"]
Jul 30 15:37:23.777222 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.777202    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:37:23.798234 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.798174    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:23.889006 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.888977    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68691566-6030-4d03-be72-58ddcad859d9-maas-api-tls\") pod \"68691566-6030-4d03-be72-58ddcad859d9\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") "
Jul 30 15:37:23.889146 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.889044    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm96p\" (UniqueName: \"kubernetes.io/projected/68691566-6030-4d03-be72-58ddcad859d9-kube-api-access-wm96p\") pod \"68691566-6030-4d03-be72-58ddcad859d9\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") "
Jul 30 15:37:23.889146 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.889069    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68691566-6030-4d03-be72-58ddcad859d9-openshift-service-ca\") pod \"68691566-6030-4d03-be72-58ddcad859d9\" (UID: \"68691566-6030-4d03-be72-58ddcad859d9\") "
Jul 30 15:37:23.889502 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.889473    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68691566-6030-4d03-be72-58ddcad859d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68691566-6030-4d03-be72-58ddcad859d9" (UID: "68691566-6030-4d03-be72-58ddcad859d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:37:23.891182 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.891158    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68691566-6030-4d03-be72-58ddcad859d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68691566-6030-4d03-be72-58ddcad859d9" (UID: "68691566-6030-4d03-be72-58ddcad859d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:37:23.891272 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.891201    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68691566-6030-4d03-be72-58ddcad859d9-kube-api-access-wm96p" (OuterVolumeSpecName: "kube-api-access-wm96p") pod "68691566-6030-4d03-be72-58ddcad859d9" (UID: "68691566-6030-4d03-be72-58ddcad859d9"). InnerVolumeSpecName "kube-api-access-wm96p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:37:23.989859 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.989819    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm96p\" (UniqueName: \"kubernetes.io/projected/68691566-6030-4d03-be72-58ddcad859d9-kube-api-access-wm96p\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:37:23.989859 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.989856    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68691566-6030-4d03-be72-58ddcad859d9-openshift-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:37:23.989859 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:23.989866    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68691566-6030-4d03-be72-58ddcad859d9-maas-api-tls\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:37:24.188587 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.188552    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:24.394310 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.394276    2582 generic.go:356] "Generic (PLEG): container finished" podID="68691566-6030-4d03-be72-58ddcad859d9" containerID="92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84" exitCode=0
Jul 30 15:37:24.394494 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.394315    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz" event={"ID":"68691566-6030-4d03-be72-58ddcad859d9","Type":"ContainerDied","Data":"92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84"}
Jul 30 15:37:24.394494 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.394362    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz" event={"ID":"68691566-6030-4d03-be72-58ddcad859d9","Type":"ContainerDied","Data":"9c00dcf7af497a199a1d9150669edba3a382574d07d86ce6b4a09121c5527633"}
Jul 30 15:37:24.394494 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.394385    2582 scope.go:117] "RemoveContainer" containerID="92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84"
Jul 30 15:37:24.394494 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.394323    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"
Jul 30 15:37:24.403213 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.403195    2582 scope.go:117] "RemoveContainer" containerID="92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84"
Jul 30 15:37:24.403527 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:37:24.403507    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84\": container with ID starting with 92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84 not found: ID does not exist" containerID="92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84"
Jul 30 15:37:24.403594 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.403536    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84"} err="failed to get container status \"92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84\": rpc error: code = NotFound desc = could not find container \"92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84\": container with ID starting with 92bfbbc589a8854db715ecba089ec76da2ea59493c76b079410fcb41b85d0f84 not found: ID does not exist"
Jul 30 15:37:24.414447 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.414424    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"]
Jul 30 15:37:24.417913 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.417895    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1c473-b5fd589df-dk2jz"]
Jul 30 15:37:24.807221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.807185    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64c9be75-30e4-4f81-91a3-8a032fd5912f" path="/var/lib/kubelet/pods/64c9be75-30e4-4f81-91a3-8a032fd5912f/volumes"
Jul 30 15:37:24.807627 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:24.807583    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68691566-6030-4d03-be72-58ddcad859d9" path="/var/lib/kubelet/pods/68691566-6030-4d03-be72-58ddcad859d9/volumes"
Jul 30 15:37:29.926430 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:29.925895    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:37:30.360574 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:30.360515    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:30.463632 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:30.463599    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:30.692130 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:30.692099    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:30.964010 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:30.963936    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:31.102413 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.102362    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"]
Jul 30 15:37:31.102681 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.102668    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68691566-6030-4d03-be72-58ddcad859d9" containerName="maas-api"
Jul 30 15:37:31.102726 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.102684    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68691566-6030-4d03-be72-58ddcad859d9" containerName="maas-api"
Jul 30 15:37:31.102726 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.102695    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c9be75-30e4-4f81-91a3-8a032fd5912f" containerName="revoke-keys"
Jul 30 15:37:31.102726 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.102701    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c9be75-30e4-4f81-91a3-8a032fd5912f" containerName="revoke-keys"
Jul 30 15:37:31.102823 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.102750    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c9be75-30e4-4f81-91a3-8a032fd5912f" containerName="revoke-keys"
Jul 30 15:37:31.102823 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.102759    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68691566-6030-4d03-be72-58ddcad859d9" containerName="maas-api"
Jul 30 15:37:31.107069 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.107052    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"
Jul 30 15:37:31.108915 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.108893    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:37:31.112347 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.112323    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"]
Jul 30 15:37:31.146429 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.146408    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc28999-69ef-4018-8f7f-cf0d783e5a64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj\" (UID: \"5bc28999-69ef-4018-8f7f-cf0d783e5a64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"
Jul 30 15:37:31.247591 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.247524    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc28999-69ef-4018-8f7f-cf0d783e5a64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj\" (UID: \"5bc28999-69ef-4018-8f7f-cf0d783e5a64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"
Jul 30 15:37:31.248059 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.248041    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc28999-69ef-4018-8f7f-cf0d783e5a64-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj\" (UID: \"5bc28999-69ef-4018-8f7f-cf0d783e5a64\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"
Jul 30 15:37:31.418148 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.418114    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"
Jul 30 15:37:31.545543 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:31.545510    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"]
Jul 30 15:37:31.548747 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:37:31.548721    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bc28999_69ef_4018_8f7f_cf0d783e5a64.slice/crio-16ab42bb960da747aa6eb065a43ff1c99d14c76907eaec385b0095eaf5989514 WatchSource:0}: Error finding container 16ab42bb960da747aa6eb065a43ff1c99d14c76907eaec385b0095eaf5989514: Status 404 returned error can't find the container with id 16ab42bb960da747aa6eb065a43ff1c99d14c76907eaec385b0095eaf5989514
Jul 30 15:37:32.429146 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:32.429109    2582 generic.go:356] "Generic (PLEG): container finished" podID="5bc28999-69ef-4018-8f7f-cf0d783e5a64" containerID="63b9de20d3d1f4e1da836ae53b0dcfa3f3b6f762c7ea6e68d2fbd2b64e91a2f3" exitCode=0
Jul 30 15:37:32.429500 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:32.429194    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj" event={"ID":"5bc28999-69ef-4018-8f7f-cf0d783e5a64","Type":"ContainerDied","Data":"63b9de20d3d1f4e1da836ae53b0dcfa3f3b6f762c7ea6e68d2fbd2b64e91a2f3"}
Jul 30 15:37:32.429500 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:32.429243    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj" event={"ID":"5bc28999-69ef-4018-8f7f-cf0d783e5a64","Type":"ContainerStarted","Data":"16ab42bb960da747aa6eb065a43ff1c99d14c76907eaec385b0095eaf5989514"}
Jul 30 15:37:33.560152 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:33.560122    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"
Jul 30 15:37:33.668008 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:33.667967    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc28999-69ef-4018-8f7f-cf0d783e5a64-maas-api-service-ca\") pod \"5bc28999-69ef-4018-8f7f-cf0d783e5a64\" (UID: \"5bc28999-69ef-4018-8f7f-cf0d783e5a64\") "
Jul 30 15:37:33.668323 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:33.668305    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bc28999-69ef-4018-8f7f-cf0d783e5a64-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5bc28999-69ef-4018-8f7f-cf0d783e5a64" (UID: "5bc28999-69ef-4018-8f7f-cf0d783e5a64"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:37:33.769390 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:33.769319    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc28999-69ef-4018-8f7f-cf0d783e5a64-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:37:34.439056 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:34.439027    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"
Jul 30 15:37:34.439056 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:34.439034    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj" event={"ID":"5bc28999-69ef-4018-8f7f-cf0d783e5a64","Type":"ContainerDied","Data":"16ab42bb960da747aa6eb065a43ff1c99d14c76907eaec385b0095eaf5989514"}
Jul 30 15:37:34.439272 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:34.439082    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16ab42bb960da747aa6eb065a43ff1c99d14c76907eaec385b0095eaf5989514"
Jul 30 15:37:46.545098 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.545009    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"]
Jul 30 15:37:46.545681 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.545244    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk" podUID="79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0" containerName="maas-api" containerID="cri-o://c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf" gracePeriod=30
Jul 30 15:37:46.576594 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.576563    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:46.654897 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.654844    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:46.750114 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.750078    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:46.783291 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.783244    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"]
Jul 30 15:37:46.783787 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.783765    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:37:46.785752 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.785728    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e2257c-x7wgj"]
Jul 30 15:37:46.807864 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.807801    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bc28999-69ef-4018-8f7f-cf0d783e5a64" path="/var/lib/kubelet/pods/5bc28999-69ef-4018-8f7f-cf0d783e5a64/volumes"
Jul 30 15:37:46.813088 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.813069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:46.875122 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.875101    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jb4k\" (UniqueName: \"kubernetes.io/projected/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-kube-api-access-2jb4k\") pod \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") "
Jul 30 15:37:46.875232 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.875147    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-openshift-service-ca\") pod \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") "
Jul 30 15:37:46.875232 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.875206    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls\") pod \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\" (UID: \"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0\") "
Jul 30 15:37:46.875550 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.875525    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0" (UID: "79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:37:46.877294 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.877273    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-kube-api-access-2jb4k" (OuterVolumeSpecName: "kube-api-access-2jb4k") pod "79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0" (UID: "79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0"). InnerVolumeSpecName "kube-api-access-2jb4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:37:46.877477 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.877455    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0" (UID: "79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:37:46.976000 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.975957    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-maas-api-tls\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:37:46.976000 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.975997    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jb4k\" (UniqueName: \"kubernetes.io/projected/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-kube-api-access-2jb4k\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:37:46.976000 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:46.976008    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0-openshift-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:37:47.232691 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:47.232660    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:47.491609 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:47.491525    2582 generic.go:356] "Generic (PLEG): container finished" podID="79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0" containerID="c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf" exitCode=0
Jul 30 15:37:47.491609 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:47.491566    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"
Jul 30 15:37:47.491609 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:47.491594    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk" event={"ID":"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0","Type":"ContainerDied","Data":"c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf"}
Jul 30 15:37:47.491840 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:47.491635    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk" event={"ID":"79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0","Type":"ContainerDied","Data":"bce32e2487201de36364022234563b97a7592860b60c16a1d38401cde8e66451"}
Jul 30 15:37:47.491840 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:47.491655    2582 scope.go:117] "RemoveContainer" containerID="c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf"
Jul 30 15:37:47.500588 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:47.500568    2582 scope.go:117] "RemoveContainer" containerID="c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf"
Jul 30 15:37:47.500857 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:37:47.500827    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf\": container with ID starting with c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf not found: ID does not exist" containerID="c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf"
Jul 30 15:37:47.500925 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:47.500870    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf"} err="failed to get container status \"c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf\": rpc error: code = NotFound desc = could not find container \"c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf\": container with ID starting with c2d71092d51d74c7a93122e451a7484aa00591a7b7b108b8cdf2b07520139caf not found: ID does not exist"
Jul 30 15:37:47.514261 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:47.514229    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"]
Jul 30 15:37:47.518180 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:47.518152    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e2257c-7cb84b55c7-x4gmk"]
Jul 30 15:37:48.806686 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:48.806655    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0" path="/var/lib/kubelet/pods/79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0/volumes"
Jul 30 15:37:52.031108 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:52.031077    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:37:53.273126 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:53.273088    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:54.101522 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:54.101479    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:54.201958 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:54.201923    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:54.301620 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:54.301585    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:54.618970 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:54.618939    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:54.999120 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:54.999088    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:57.978719 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:57.978689    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:58.149596 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:58.149555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:59.113182 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:59.113149    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:59.206970 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:59.206933    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:59.701593 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:59.701566    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:37:59.801002 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:37:59.800972    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:03.126268 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:03.126229    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:38:23.156361 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:23.156325    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:23.521734 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:23.521700    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:23.833767 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:23.833691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:24.203154 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:24.203124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:27.101573 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:27.101542    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:27.200812 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:27.200769    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:28.290292 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.290262    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"]
Jul 30 15:38:28.290674 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.290554    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0" containerName="maas-api"
Jul 30 15:38:28.290674 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.290565    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0" containerName="maas-api"
Jul 30 15:38:28.290674 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.290586    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bc28999-69ef-4018-8f7f-cf0d783e5a64" containerName="revoke-keys"
Jul 30 15:38:28.290674 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.290592    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bc28999-69ef-4018-8f7f-cf0d783e5a64" containerName="revoke-keys"
Jul 30 15:38:28.290674 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.290635    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bc28999-69ef-4018-8f7f-cf0d783e5a64" containerName="revoke-keys"
Jul 30 15:38:28.290674 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.290642    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ea5f9b-7ae2-4a10-b78c-b5a7779fc8b0" containerName="maas-api"
Jul 30 15:38:28.293539 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.293523    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.295557 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.295536    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-2b1f1f-serving-cert\""
Jul 30 15:38:28.295694 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.295570    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 15:38:28.295694 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.295663    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 15:38:28.295938 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.295921    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q7854\""
Jul 30 15:38:28.303369 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.303349    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"]
Jul 30 15:38:28.326038 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.326006    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:28.375827 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.375794    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkghz\" (UniqueName: \"kubernetes.io/projected/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-kube-api-access-bkghz\") pod \"maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.375996 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.375832    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.375996 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.375875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.476252 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.476217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.476478 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.476290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkghz\" (UniqueName: \"kubernetes.io/projected/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-kube-api-access-bkghz\") pod \"maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.476478 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.476317    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.477025 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.477004    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.478842 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.478825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.481388 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.481366    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:28.484045 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.484029    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkghz\" (UniqueName: \"kubernetes.io/projected/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-kube-api-access-bkghz\") pod \"maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.604033 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.603956    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:28.734064 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.733936    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"]
Jul 30 15:38:28.737205 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:38:28.737176    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04052c3_5a6f_4312_bc9b_a8fc05900cc8.slice/crio-322e8aa499e34d751efe20ecdfd53bfe8e94feac176b643db513dbd06d60fbe0 WatchSource:0}: Error finding container 322e8aa499e34d751efe20ecdfd53bfe8e94feac176b643db513dbd06d60fbe0: Status 404 returned error can't find the container with id 322e8aa499e34d751efe20ecdfd53bfe8e94feac176b643db513dbd06d60fbe0
Jul 30 15:38:28.836574 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.836538    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:28.905974 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:28.905883    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:38:29.672180 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:29.672088    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv" event={"ID":"a04052c3-5a6f-4312-bc9b-a8fc05900cc8","Type":"ContainerStarted","Data":"a7f50c4f114c1ac5de5d6adb05a05232b99b3d2a3e83a79aa4fc58413b495cca"}
Jul 30 15:38:29.672180 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:29.672150    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv" event={"ID":"a04052c3-5a6f-4312-bc9b-a8fc05900cc8","Type":"ContainerStarted","Data":"322e8aa499e34d751efe20ecdfd53bfe8e94feac176b643db513dbd06d60fbe0"}
Jul 30 15:38:29.672603 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:29.672215    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:29.688582 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:29.688542    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv" podStartSLOduration=1.173427614 podStartE2EDuration="1.688527211s" totalImagesPullingTime="515.099597ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:38:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:38:29.68793079 +0000 UTC m=+3015.361376905" watchObservedRunningTime="2026-07-30 15:38:29.688527211 +0000 UTC m=+3015.361973324"
Jul 30 15:38:34.822337 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:34.822304    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:38:35.682156 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:35.682124    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:38:52.187653 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:38:52.187603    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:39:00.183731 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:00.183688    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:39:03.964283 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:03.964245    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:39:26.027149 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.027115    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"]
Jul 30 15:39:26.030090 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.030074    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.032646 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.032618    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2b1f1f\"/\"kube-root-ca.crt\""
Jul 30 15:39:26.032646 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.032642    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2b1f1f\"/\"openshift-service-ca.crt\""
Jul 30 15:39:26.032823 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.032618    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2b1f1f\"/\"auth-test-model-2b1f1f-kserve-self-signed-certs\""
Jul 30 15:39:26.032823 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.032663    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2b1f1f\"/\"default-dockercfg-fc948\""
Jul 30 15:39:26.039742 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.039721    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"]
Jul 30 15:39:26.059224 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.059203    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:39:26.089936 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.089905    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-dshm\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.090050 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.089938    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-tmp-dir\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.090050 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.089961    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-home\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.090050 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.089983    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/167ac933-c64f-43be-b457-22a04587974e-tls-certs\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.090050 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.090040    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb8nt\" (UniqueName: \"kubernetes.io/projected/167ac933-c64f-43be-b457-22a04587974e-kube-api-access-lb8nt\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.090186 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.090070    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-model-cache\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.191495 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.191467    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-dshm\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.191495 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.191498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-tmp-dir\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.191701 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.191522    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-home\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.191701 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.191538    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/167ac933-c64f-43be-b457-22a04587974e-tls-certs\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.191701 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.191576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb8nt\" (UniqueName: \"kubernetes.io/projected/167ac933-c64f-43be-b457-22a04587974e-kube-api-access-lb8nt\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.191701 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.191613    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-model-cache\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.191985 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.191964    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-tmp-dir\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.192044 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.192009    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-home\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.192044 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.192025    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-model-cache\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.194218 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.194190    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-dshm\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.194759 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.194734    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/167ac933-c64f-43be-b457-22a04587974e-tls-certs\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.198755 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.198731    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb8nt\" (UniqueName: \"kubernetes.io/projected/167ac933-c64f-43be-b457-22a04587974e-kube-api-access-lb8nt\") pod \"auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") " pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.340152 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.340085    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:26.470477 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.470450    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"]
Jul 30 15:39:26.476609 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:39:26.476559    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod167ac933_c64f_43be_b457_22a04587974e.slice/crio-4fb808cfc018ef992ed5674b7a54f80935df60a0dbdc5fd1a5f381e7a94c184a WatchSource:0}: Error finding container 4fb808cfc018ef992ed5674b7a54f80935df60a0dbdc5fd1a5f381e7a94c184a: Status 404 returned error can't find the container with id 4fb808cfc018ef992ed5674b7a54f80935df60a0dbdc5fd1a5f381e7a94c184a
Jul 30 15:39:26.913371 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.913338    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r" event={"ID":"167ac933-c64f-43be-b457-22a04587974e","Type":"ContainerStarted","Data":"a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63"}
Jul 30 15:39:26.913371 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.913377    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r" event={"ID":"167ac933-c64f-43be-b457-22a04587974e","Type":"ContainerStarted","Data":"4fb808cfc018ef992ed5674b7a54f80935df60a0dbdc5fd1a5f381e7a94c184a"}
Jul 30 15:39:26.936139 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:26.936086    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r" podStartSLOduration=0.936070094 podStartE2EDuration="936.070094ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:39:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:39:26.929103446 +0000 UTC m=+3072.602549539" watchObservedRunningTime="2026-07-30 15:39:26.936070094 +0000 UTC m=+3072.609516207"
Jul 30 15:39:30.026047 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:30.026011    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:39:36.340378 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:36.340344    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:36.341023 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:36.340410    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:36.352949 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:36.352925    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:36.964670 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:36.964642    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:39:53.623575 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:39:53.623535    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:40:19.816794 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:40:19.816754    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:40:26.755251 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:40:26.755217    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:40:38.522683 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:40:38.522595    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:41:04.149662 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:41:04.149626    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:41:05.131970 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:41:05.131934    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:41:06.468566 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:41:06.468530    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:41:16.821381 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:41:16.821346    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:41:20.930216 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:41:20.930183    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:41:48.324471 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:41:48.324430    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:42:05.025714 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:42:05.025681    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:42:15.030517 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:42:15.030438    2582 scope.go:117] "RemoveContainer" containerID="851c6a6270234691a9e5932c12b0e8232e4d1000d863a57452a5f8846105260e"
Jul 30 15:42:15.038287 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:42:15.038269    2582 scope.go:117] "RemoveContainer" containerID="1f2e2c2491313baf3b2cb5a40ba5d2dbe7cb1cccb32550698dd104c3c052d24f"
Jul 30 15:42:30.519497 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:42:30.519460    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:42:49.132729 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:42:49.132691    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:43:09.593651 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:43:09.593615    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:43:09.678599 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:43:09.678571    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:43:15.059866 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:43:15.059833    2582 scope.go:117] "RemoveContainer" containerID="0e5c9db0139c5d4a94f7d0329ef8e7c03cae6e09b27c6ad232906554b9db9fbf"
Jul 30 15:43:15.838812 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:43:15.838775    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:43:16.637045 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:43:16.636998    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:43:33.119716 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:43:33.119683    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:43:45.027696 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:43:45.027614    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:43:45.648697 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:43:45.648667    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:44:13.334378 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:13.334346    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:44:15.076685 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:15.076658    2582 scope.go:117] "RemoveContainer" containerID="63b9de20d3d1f4e1da836ae53b0dcfa3f3b6f762c7ea6e68d2fbd2b64e91a2f3"
Jul 30 15:44:18.266721 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:18.266684    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:44:19.304705 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:19.304674    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"]
Jul 30 15:44:19.326634 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:19.326595    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:44:30.655225 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:30.655189    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:44:30.820586 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:30.820551    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:44:34.323810 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.323749    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r" podUID="167ac933-c64f-43be-b457-22a04587974e" containerName="main" containerID="cri-o://a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63" gracePeriod=45
Jul 30 15:44:34.570118 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.570094    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:44:34.618591 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618521    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-model-cache\") pod \"167ac933-c64f-43be-b457-22a04587974e\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") "
Jul 30 15:44:34.618591 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618562    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb8nt\" (UniqueName: \"kubernetes.io/projected/167ac933-c64f-43be-b457-22a04587974e-kube-api-access-lb8nt\") pod \"167ac933-c64f-43be-b457-22a04587974e\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") "
Jul 30 15:44:34.618787 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618628    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-tmp-dir\") pod \"167ac933-c64f-43be-b457-22a04587974e\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") "
Jul 30 15:44:34.618787 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618644    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-dshm\") pod \"167ac933-c64f-43be-b457-22a04587974e\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") "
Jul 30 15:44:34.618787 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618658    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-home\") pod \"167ac933-c64f-43be-b457-22a04587974e\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") "
Jul 30 15:44:34.618787 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618682    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/167ac933-c64f-43be-b457-22a04587974e-tls-certs\") pod \"167ac933-c64f-43be-b457-22a04587974e\" (UID: \"167ac933-c64f-43be-b457-22a04587974e\") "
Jul 30 15:44:34.618787 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618765    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-model-cache" (OuterVolumeSpecName: "model-cache") pod "167ac933-c64f-43be-b457-22a04587974e" (UID: "167ac933-c64f-43be-b457-22a04587974e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:44:34.619046 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618836    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "167ac933-c64f-43be-b457-22a04587974e" (UID: "167ac933-c64f-43be-b457-22a04587974e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:44:34.619046 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618936    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-tmp-dir\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:44:34.619046 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618952    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-model-cache\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:44:34.619046 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.618964    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-home" (OuterVolumeSpecName: "home") pod "167ac933-c64f-43be-b457-22a04587974e" (UID: "167ac933-c64f-43be-b457-22a04587974e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:44:34.621040 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.621016    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/167ac933-c64f-43be-b457-22a04587974e-kube-api-access-lb8nt" (OuterVolumeSpecName: "kube-api-access-lb8nt") pod "167ac933-c64f-43be-b457-22a04587974e" (UID: "167ac933-c64f-43be-b457-22a04587974e"). InnerVolumeSpecName "kube-api-access-lb8nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:44:34.621040 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.621014    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-dshm" (OuterVolumeSpecName: "dshm") pod "167ac933-c64f-43be-b457-22a04587974e" (UID: "167ac933-c64f-43be-b457-22a04587974e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:44:34.621165 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.621084    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/167ac933-c64f-43be-b457-22a04587974e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "167ac933-c64f-43be-b457-22a04587974e" (UID: "167ac933-c64f-43be-b457-22a04587974e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:44:34.720141 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.720116    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/167ac933-c64f-43be-b457-22a04587974e-tls-certs\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:44:34.720141 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.720139    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb8nt\" (UniqueName: \"kubernetes.io/projected/167ac933-c64f-43be-b457-22a04587974e-kube-api-access-lb8nt\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:44:34.720286 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.720150    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-dshm\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:44:34.720286 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:34.720159    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/167ac933-c64f-43be-b457-22a04587974e-home\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:44:35.173034 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:35.173005    2582 generic.go:356] "Generic (PLEG): container finished" podID="167ac933-c64f-43be-b457-22a04587974e" containerID="a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63" exitCode=0
Jul 30 15:44:35.173236 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:35.173048    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"
Jul 30 15:44:35.173236 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:35.173103    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r" event={"ID":"167ac933-c64f-43be-b457-22a04587974e","Type":"ContainerDied","Data":"a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63"}
Jul 30 15:44:35.173236 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:35.173154    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r" event={"ID":"167ac933-c64f-43be-b457-22a04587974e","Type":"ContainerDied","Data":"4fb808cfc018ef992ed5674b7a54f80935df60a0dbdc5fd1a5f381e7a94c184a"}
Jul 30 15:44:35.173236 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:35.173178    2582 scope.go:117] "RemoveContainer" containerID="a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63"
Jul 30 15:44:35.181461 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:35.181442    2582 scope.go:117] "RemoveContainer" containerID="a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63"
Jul 30 15:44:35.181713 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:44:35.181695    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63\": container with ID starting with a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63 not found: ID does not exist" containerID="a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63"
Jul 30 15:44:35.181753 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:35.181722    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63"} err="failed to get container status \"a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63\": rpc error: code = NotFound desc = could not find container \"a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63\": container with ID starting with a51a8a3d8a2bbda8476924cdc529c2c9ca22407b13baf70cbadeed125c921f63 not found: ID does not exist"
Jul 30 15:44:35.188136 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:35.188115    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"]
Jul 30 15:44:35.190457 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:35.190437    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/auth-test-model-2b1f1f-kserve-78d69d8fbc-cv78r"]
Jul 30 15:44:36.806604 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:36.806573    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="167ac933-c64f-43be-b457-22a04587974e" path="/var/lib/kubelet/pods/167ac933-c64f-43be-b457-22a04587974e/volumes"
Jul 30 15:44:41.129473 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:44:41.129440    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:45:00.157811 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.157778    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757105-74882"]
Jul 30 15:45:00.158268 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.158035    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="167ac933-c64f-43be-b457-22a04587974e" containerName="main"
Jul 30 15:45:00.158268 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.158046    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="167ac933-c64f-43be-b457-22a04587974e" containerName="main"
Jul 30 15:45:00.158268 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.158105    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="167ac933-c64f-43be-b457-22a04587974e" containerName="main"
Jul 30 15:45:00.162748 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.161984    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757105-74882"
Jul 30 15:45:00.165624 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.165593    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:45:00.166606 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.166588    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22"]
Jul 30 15:45:00.169539 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.169521    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z"]
Jul 30 15:45:00.169664 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.169631    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22"
Jul 30 15:45:00.174026 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.174002    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757105-74882"
Jul 30 15:45:00.174830 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.174808    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757105-74882"]
Jul 30 15:45:00.174956 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.174896    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z"
Jul 30 15:45:00.179885 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.179867    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z"]
Jul 30 15:45:00.185007 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.184983    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22"]
Jul 30 15:45:00.207244 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.207224    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22"
Jul 30 15:45:00.212154 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.212135    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z"
Jul 30 15:45:00.349040 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.348971    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757105-74882"]
Jul 30 15:45:00.354178 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:45:00.354149    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4fa00f2_3a30_4925_8564_1d1c02d74e84.slice/crio-6586d7e66d679b384d5961f5292dc92309150e5d85b56054005b47ff5862d7f8 WatchSource:0}: Error finding container 6586d7e66d679b384d5961f5292dc92309150e5d85b56054005b47ff5862d7f8: Status 404 returned error can't find the container with id 6586d7e66d679b384d5961f5292dc92309150e5d85b56054005b47ff5862d7f8
Jul 30 15:45:00.376860 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.376834    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22"]
Jul 30 15:45:00.389483 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:45:00.389456    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464dfab2_2ee7_42c4_b47f_59a83519dc95.slice/crio-45c675e3385a5d1ad58f4db1733d0c1c4a88118315f6bba10b0532d9ec831319 WatchSource:0}: Error finding container 45c675e3385a5d1ad58f4db1733d0c1c4a88118315f6bba10b0532d9ec831319: Status 404 returned error can't find the container with id 45c675e3385a5d1ad58f4db1733d0c1c4a88118315f6bba10b0532d9ec831319
Jul 30 15:45:00.399054 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:00.399031    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z"]
Jul 30 15:45:00.401337 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:45:00.401311    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod424ab002_57a5_414a_a3bc_3eb5d6681ef4.slice/crio-6bf86c91c0801a0b6ea9724629a9c8b03ee17ca488b38872330aea53dabbf21f WatchSource:0}: Error finding container 6bf86c91c0801a0b6ea9724629a9c8b03ee17ca488b38872330aea53dabbf21f: Status 404 returned error can't find the container with id 6bf86c91c0801a0b6ea9724629a9c8b03ee17ca488b38872330aea53dabbf21f
Jul 30 15:45:01.278146 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:01.278119    2582 generic.go:356] "Generic (PLEG): container finished" podID="b4fa00f2-3a30-4925-8564-1d1c02d74e84" containerID="1959212bf678cb29a2ed4a1d35096113b81fc6c7f3635994c96ee06952c57c68" exitCode=0
Jul 30 15:45:01.278514 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:01.278208    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757105-74882" event={"ID":"b4fa00f2-3a30-4925-8564-1d1c02d74e84","Type":"ContainerDied","Data":"1959212bf678cb29a2ed4a1d35096113b81fc6c7f3635994c96ee06952c57c68"}
Jul 30 15:45:01.278514 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:01.278260    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757105-74882" event={"ID":"b4fa00f2-3a30-4925-8564-1d1c02d74e84","Type":"ContainerStarted","Data":"6586d7e66d679b384d5961f5292dc92309150e5d85b56054005b47ff5862d7f8"}
Jul 30 15:45:01.279611 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:01.279593    2582 generic.go:356] "Generic (PLEG): container finished" podID="464dfab2-2ee7-42c4-b47f-59a83519dc95" containerID="58a1f07b83267a1f0c386183c550598ae1d23076a9cf94e909a7ec41585c3996" exitCode=0
Jul 30 15:45:01.279702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:01.279665    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22" event={"ID":"464dfab2-2ee7-42c4-b47f-59a83519dc95","Type":"ContainerDied","Data":"58a1f07b83267a1f0c386183c550598ae1d23076a9cf94e909a7ec41585c3996"}
Jul 30 15:45:01.279751 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:01.279703    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22" event={"ID":"464dfab2-2ee7-42c4-b47f-59a83519dc95","Type":"ContainerStarted","Data":"45c675e3385a5d1ad58f4db1733d0c1c4a88118315f6bba10b0532d9ec831319"}
Jul 30 15:45:01.281016 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:01.280997    2582 generic.go:356] "Generic (PLEG): container finished" podID="424ab002-57a5-414a-a3bc-3eb5d6681ef4" containerID="8c8d0902a8643c38514fcb47123086c63e006805e85057ad1578b4c3625ff707" exitCode=0
Jul 30 15:45:01.281109 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:01.281026    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z" event={"ID":"424ab002-57a5-414a-a3bc-3eb5d6681ef4","Type":"ContainerDied","Data":"8c8d0902a8643c38514fcb47123086c63e006805e85057ad1578b4c3625ff707"}
Jul 30 15:45:01.281109 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:01.281066    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z" event={"ID":"424ab002-57a5-414a-a3bc-3eb5d6681ef4","Type":"ContainerStarted","Data":"6bf86c91c0801a0b6ea9724629a9c8b03ee17ca488b38872330aea53dabbf21f"}
Jul 30 15:45:02.418267 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:02.418235    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757105-74882"
Jul 30 15:45:02.450706 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:02.450682    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22"
Jul 30 15:45:02.453537 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:02.453513    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z"
Jul 30 15:45:03.291634 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:03.291601    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z"
Jul 30 15:45:03.291634 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:03.291616    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z" event={"ID":"424ab002-57a5-414a-a3bc-3eb5d6681ef4","Type":"ContainerDied","Data":"6bf86c91c0801a0b6ea9724629a9c8b03ee17ca488b38872330aea53dabbf21f"}
Jul 30 15:45:03.291906 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:03.291700    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bf86c91c0801a0b6ea9724629a9c8b03ee17ca488b38872330aea53dabbf21f"
Jul 30 15:45:03.292829 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:03.292806    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757105-74882"
Jul 30 15:45:03.292944 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:03.292833    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757105-74882" event={"ID":"b4fa00f2-3a30-4925-8564-1d1c02d74e84","Type":"ContainerDied","Data":"6586d7e66d679b384d5961f5292dc92309150e5d85b56054005b47ff5862d7f8"}
Jul 30 15:45:03.292944 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:03.292870    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6586d7e66d679b384d5961f5292dc92309150e5d85b56054005b47ff5862d7f8"
Jul 30 15:45:03.294177 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:03.294157    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22" event={"ID":"464dfab2-2ee7-42c4-b47f-59a83519dc95","Type":"ContainerDied","Data":"45c675e3385a5d1ad58f4db1733d0c1c4a88118315f6bba10b0532d9ec831319"}
Jul 30 15:45:03.294286 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:03.294190    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45c675e3385a5d1ad58f4db1733d0c1c4a88118315f6bba10b0532d9ec831319"
Jul 30 15:45:03.294286 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:03.294165    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22"
Jul 30 15:45:05.129333 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.129297    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"]
Jul 30 15:45:05.129702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.129591    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="424ab002-57a5-414a-a3bc-3eb5d6681ef4" containerName="cleanup"
Jul 30 15:45:05.129702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.129603    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="424ab002-57a5-414a-a3bc-3eb5d6681ef4" containerName="cleanup"
Jul 30 15:45:05.129702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.129614    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="464dfab2-2ee7-42c4-b47f-59a83519dc95" containerName="cleanup"
Jul 30 15:45:05.129702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.129619    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="464dfab2-2ee7-42c4-b47f-59a83519dc95" containerName="cleanup"
Jul 30 15:45:05.129702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.129631    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4fa00f2-3a30-4925-8564-1d1c02d74e84" containerName="cleanup"
Jul 30 15:45:05.129702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.129636    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4fa00f2-3a30-4925-8564-1d1c02d74e84" containerName="cleanup"
Jul 30 15:45:05.129702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.129687    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="424ab002-57a5-414a-a3bc-3eb5d6681ef4" containerName="cleanup"
Jul 30 15:45:05.129702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.129695    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="464dfab2-2ee7-42c4-b47f-59a83519dc95" containerName="cleanup"
Jul 30 15:45:05.129702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.129701    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4fa00f2-3a30-4925-8564-1d1c02d74e84" containerName="cleanup"
Jul 30 15:45:05.132571 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.132553    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.135841 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.135821    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2b1f1f\"/\"openshift-service-ca.crt\""
Jul 30 15:45:05.136040 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.136026    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2b1f1f\"/\"kube-root-ca.crt\""
Jul 30 15:45:05.136125 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.136107    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2b1f1f\"/\"default-dockercfg-fc948\""
Jul 30 15:45:05.136177 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.136117    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2b1f1f\"/\"sub-test-model-2b1f1f-kserve-self-signed-certs\""
Jul 30 15:45:05.142881 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.142858    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"]
Jul 30 15:45:05.169640 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.169604    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:45:05.250777 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.250738    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkjz5\" (UniqueName: \"kubernetes.io/projected/1e0c6dbd-6df0-463b-83a1-0842573ea513-kube-api-access-tkjz5\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.250969 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.250786    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-home\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.250969 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.250876    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0c6dbd-6df0-463b-83a1-0842573ea513-tls-certs\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.250969 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.250918    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-tmp-dir\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.250969 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.250961    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-model-cache\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.251109 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.250999    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-dshm\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.351842 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.351802    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-tmp-dir\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.351842 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.351844    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-model-cache\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.352068 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.351866    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-dshm\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.352068 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.351996    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkjz5\" (UniqueName: \"kubernetes.io/projected/1e0c6dbd-6df0-463b-83a1-0842573ea513-kube-api-access-tkjz5\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.352068 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.352043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-home\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.352200 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.352104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0c6dbd-6df0-463b-83a1-0842573ea513-tls-certs\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.352251 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.352236    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-model-cache\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.352339 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.352318    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-tmp-dir\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.352459 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.352439    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-home\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.354305 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.354288    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-dshm\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.354574 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.354559    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0c6dbd-6df0-463b-83a1-0842573ea513-tls-certs\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.362963 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.362942    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkjz5\" (UniqueName: \"kubernetes.io/projected/1e0c6dbd-6df0-463b-83a1-0842573ea513-kube-api-access-tkjz5\") pod \"sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") " pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.442438 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.442383    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:05.579645 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:05.579616    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"]
Jul 30 15:45:05.582084 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:45:05.582059    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e0c6dbd_6df0_463b_83a1_0842573ea513.slice/crio-d8d67b63df1acb69e8259ecf737d64fadadb59c516840ba13c9689770a2f2777 WatchSource:0}: Error finding container d8d67b63df1acb69e8259ecf737d64fadadb59c516840ba13c9689770a2f2777: Status 404 returned error can't find the container with id d8d67b63df1acb69e8259ecf737d64fadadb59c516840ba13c9689770a2f2777
Jul 30 15:45:06.307968 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:06.307933    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd" event={"ID":"1e0c6dbd-6df0-463b-83a1-0842573ea513","Type":"ContainerStarted","Data":"ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935"}
Jul 30 15:45:06.308377 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:06.307979    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd" event={"ID":"1e0c6dbd-6df0-463b-83a1-0842573ea513","Type":"ContainerStarted","Data":"d8d67b63df1acb69e8259ecf737d64fadadb59c516840ba13c9689770a2f2777"}
Jul 30 15:45:06.327834 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:06.327783    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd" podStartSLOduration=1.327767796 podStartE2EDuration="1.327767796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:45:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:45:06.32685504 +0000 UTC m=+3412.000301152" watchObservedRunningTime="2026-07-30 15:45:06.327767796 +0000 UTC m=+3412.001213886"
Jul 30 15:45:09.460941 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:09.460907    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:45:15.443480 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:15.443449    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:15.443924 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:15.443500    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:15.456055 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:15.456033    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:45:16.366189 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:45:16.366154    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:46:04.434643 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:46:04.434607    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:46:21.430675 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:46:21.430642    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:46:45.433948 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:46:45.433864    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:46:46.139573 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:46:46.139544    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:46:51.204596 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:46:51.204541    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:46:52.244498 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:46:52.244462    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"]
Jul 30 15:46:52.267165 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:46:52.267120    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:47:05.736479 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.736446    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"]
Jul 30 15:47:05.739467 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.739446    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.742203 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.742182    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-845a26\"/\"default-dockercfg-bc8rv\""
Jul 30 15:47:05.742333 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.742182    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-845a26\"/\"kube-root-ca.crt\""
Jul 30 15:47:05.742333 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.742183    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-845a26\"/\"openshift-service-ca.crt\""
Jul 30 15:47:05.742333 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.742183    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-845a26\"/\"rate-test-model-845a26-kserve-self-signed-certs\""
Jul 30 15:47:05.748475 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.748452    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"]
Jul 30 15:47:05.772173 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.772145    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:47:05.853005 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.852975    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-model-cache\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.853005 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.853009    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tmp-dir\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.853206 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.853046    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-dshm\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.853206 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.853107    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-home\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.853206 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.853140    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkv4m\" (UniqueName: \"kubernetes.io/projected/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-kube-api-access-jkv4m\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.853206 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.853161    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tls-certs\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.874100 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.874073    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:47:05.953998 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.953963    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-dshm\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.954204 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.954019    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-home\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.954204 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.954161    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkv4m\" (UniqueName: \"kubernetes.io/projected/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-kube-api-access-jkv4m\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.954292 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.954202    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tls-certs\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.954292 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.954236    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-model-cache\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.954292 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.954262    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tmp-dir\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.954479 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.954433    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-home\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.954630 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.954604    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tmp-dir\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.954630 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.954624    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-model-cache\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.956591 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.956570    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-dshm\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.956922 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.956901    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tls-certs\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:05.973241 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:05.973221    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkv4m\" (UniqueName: \"kubernetes.io/projected/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-kube-api-access-jkv4m\") pod \"rate-test-model-845a26-kserve-795c45c4dc-s6kcm\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") " pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:06.049662 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:06.049593    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:06.187159 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:06.187133    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"]
Jul 30 15:47:06.189780 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:47:06.189754    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod032dafa8_1b8a_4a28_bc38_b7b5808b1d77.slice/crio-16e4f6307097c6dcf586dfbd9a847625551beb420c2854c9559b543a58efdd96 WatchSource:0}: Error finding container 16e4f6307097c6dcf586dfbd9a847625551beb420c2854c9559b543a58efdd96: Status 404 returned error can't find the container with id 16e4f6307097c6dcf586dfbd9a847625551beb420c2854c9559b543a58efdd96
Jul 30 15:47:06.823528 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:06.823488    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm" event={"ID":"032dafa8-1b8a-4a28-bc38-b7b5808b1d77","Type":"ContainerStarted","Data":"8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5"}
Jul 30 15:47:06.823528 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:06.823531    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm" event={"ID":"032dafa8-1b8a-4a28-bc38-b7b5808b1d77","Type":"ContainerStarted","Data":"16e4f6307097c6dcf586dfbd9a847625551beb420c2854c9559b543a58efdd96"}
Jul 30 15:47:06.842481 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:06.842435    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm" podStartSLOduration=1.8424210300000001 podStartE2EDuration="1.84242103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:47:06.841234232 +0000 UTC m=+3532.514680369" watchObservedRunningTime="2026-07-30 15:47:06.84242103 +0000 UTC m=+3532.515867139"
Jul 30 15:47:07.263525 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.263488    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd" podUID="1e0c6dbd-6df0-463b-83a1-0842573ea513" containerName="main" containerID="cri-o://ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935" gracePeriod=45
Jul 30 15:47:07.509800 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.509771    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:47:07.568227 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.568147    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-tmp-dir\") pod \"1e0c6dbd-6df0-463b-83a1-0842573ea513\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") "
Jul 30 15:47:07.568227 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.568185    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkjz5\" (UniqueName: \"kubernetes.io/projected/1e0c6dbd-6df0-463b-83a1-0842573ea513-kube-api-access-tkjz5\") pod \"1e0c6dbd-6df0-463b-83a1-0842573ea513\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") "
Jul 30 15:47:07.568227 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.568217    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-home\") pod \"1e0c6dbd-6df0-463b-83a1-0842573ea513\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") "
Jul 30 15:47:07.568533 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.568247    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-dshm\") pod \"1e0c6dbd-6df0-463b-83a1-0842573ea513\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") "
Jul 30 15:47:07.568533 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.568282    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0c6dbd-6df0-463b-83a1-0842573ea513-tls-certs\") pod \"1e0c6dbd-6df0-463b-83a1-0842573ea513\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") "
Jul 30 15:47:07.568533 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.568321    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-model-cache\") pod \"1e0c6dbd-6df0-463b-83a1-0842573ea513\" (UID: \"1e0c6dbd-6df0-463b-83a1-0842573ea513\") "
Jul 30 15:47:07.568533 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.568490    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1e0c6dbd-6df0-463b-83a1-0842573ea513" (UID: "1e0c6dbd-6df0-463b-83a1-0842573ea513"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:47:07.568678 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.568563    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-home" (OuterVolumeSpecName: "home") pod "1e0c6dbd-6df0-463b-83a1-0842573ea513" (UID: "1e0c6dbd-6df0-463b-83a1-0842573ea513"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:47:07.568678 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.568639    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-model-cache" (OuterVolumeSpecName: "model-cache") pod "1e0c6dbd-6df0-463b-83a1-0842573ea513" (UID: "1e0c6dbd-6df0-463b-83a1-0842573ea513"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:47:07.570480 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.570459    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-dshm" (OuterVolumeSpecName: "dshm") pod "1e0c6dbd-6df0-463b-83a1-0842573ea513" (UID: "1e0c6dbd-6df0-463b-83a1-0842573ea513"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:47:07.570595 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.570487    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e0c6dbd-6df0-463b-83a1-0842573ea513-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1e0c6dbd-6df0-463b-83a1-0842573ea513" (UID: "1e0c6dbd-6df0-463b-83a1-0842573ea513"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:47:07.570595 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.570532    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e0c6dbd-6df0-463b-83a1-0842573ea513-kube-api-access-tkjz5" (OuterVolumeSpecName: "kube-api-access-tkjz5") pod "1e0c6dbd-6df0-463b-83a1-0842573ea513" (UID: "1e0c6dbd-6df0-463b-83a1-0842573ea513"). InnerVolumeSpecName "kube-api-access-tkjz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:47:07.669732 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.669697    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-model-cache\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:47:07.669732 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.669726    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-tmp-dir\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:47:07.669732 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.669738    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkjz5\" (UniqueName: \"kubernetes.io/projected/1e0c6dbd-6df0-463b-83a1-0842573ea513-kube-api-access-tkjz5\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:47:07.669972 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.669747    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-home\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:47:07.669972 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.669754    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e0c6dbd-6df0-463b-83a1-0842573ea513-dshm\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:47:07.669972 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.669761    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e0c6dbd-6df0-463b-83a1-0842573ea513-tls-certs\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:47:07.829587 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.829497    2582 generic.go:356] "Generic (PLEG): container finished" podID="1e0c6dbd-6df0-463b-83a1-0842573ea513" containerID="ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935" exitCode=0
Jul 30 15:47:07.829587 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.829544    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"
Jul 30 15:47:07.830104 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.829584    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd" event={"ID":"1e0c6dbd-6df0-463b-83a1-0842573ea513","Type":"ContainerDied","Data":"ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935"}
Jul 30 15:47:07.830104 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.829619    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd" event={"ID":"1e0c6dbd-6df0-463b-83a1-0842573ea513","Type":"ContainerDied","Data":"d8d67b63df1acb69e8259ecf737d64fadadb59c516840ba13c9689770a2f2777"}
Jul 30 15:47:07.830104 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.829636    2582 scope.go:117] "RemoveContainer" containerID="ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935"
Jul 30 15:47:07.838006 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.837992    2582 scope.go:117] "RemoveContainer" containerID="ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935"
Jul 30 15:47:07.838254 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:47:07.838235    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935\": container with ID starting with ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935 not found: ID does not exist" containerID="ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935"
Jul 30 15:47:07.838315 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.838262    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935"} err="failed to get container status \"ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935\": rpc error: code = NotFound desc = could not find container \"ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935\": container with ID starting with ffa21ea1d7244feb2d531661aa5f851ed698a7e707617bcc6c8c6eab9002d935 not found: ID does not exist"
Jul 30 15:47:07.848998 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.848976    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"]
Jul 30 15:47:07.857250 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:07.855482    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2b1f1f/sub-test-model-2b1f1f-kserve-7b484bb567-r9rtd"]
Jul 30 15:47:08.806823 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:08.806791    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e0c6dbd-6df0-463b-83a1-0842573ea513" path="/var/lib/kubelet/pods/1e0c6dbd-6df0-463b-83a1-0842573ea513/volumes"
Jul 30 15:47:15.437430 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:15.437370    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:47:16.050639 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:16.050600    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:16.050639 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:16.050647    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:16.063537 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:16.063517    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:16.877881 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:16.877856    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:47:40.866081 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:40.866048    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:47:45.021485 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:47:45.021457    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:48:42.230210 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:48:42.230169    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:48:56.728270 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:48:56.728234    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:49:26.928631 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:26.928595    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:49:32.899420 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:32.899359    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"]
Jul 30 15:49:32.979743 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:32.979718    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:33.831186 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:33.831159    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:38.226769 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.226690    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7548-139703280694576"]
Jul 30 15:49:38.227126 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.226991    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e0c6dbd-6df0-463b-83a1-0842573ea513" containerName="main"
Jul 30 15:49:38.227126 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.227002    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e0c6dbd-6df0-463b-83a1-0842573ea513" containerName="main"
Jul 30 15:49:38.227126 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.227050    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e0c6dbd-6df0-463b-83a1-0842573ea513" containerName="main"
Jul 30 15:49:38.229742 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.229726    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280694576"
Jul 30 15:49:38.232178 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.232152    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kpckp\""
Jul 30 15:49:38.235111 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.235087    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7548-139703280694576"]
Jul 30 15:49:38.325645 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.325608    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hd5h\" (UniqueName: \"kubernetes.io/projected/9fc04d30-2d55-41e9-9d42-9ba138d16abe-kube-api-access-8hd5h\") pod \"test-curl-7548-139703280694576\" (UID: \"9fc04d30-2d55-41e9-9d42-9ba138d16abe\") " pod="opendatahub/test-curl-7548-139703280694576"
Jul 30 15:49:38.426527 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.426492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hd5h\" (UniqueName: \"kubernetes.io/projected/9fc04d30-2d55-41e9-9d42-9ba138d16abe-kube-api-access-8hd5h\") pod \"test-curl-7548-139703280694576\" (UID: \"9fc04d30-2d55-41e9-9d42-9ba138d16abe\") " pod="opendatahub/test-curl-7548-139703280694576"
Jul 30 15:49:38.435008 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.434978    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hd5h\" (UniqueName: \"kubernetes.io/projected/9fc04d30-2d55-41e9-9d42-9ba138d16abe-kube-api-access-8hd5h\") pod \"test-curl-7548-139703280694576\" (UID: \"9fc04d30-2d55-41e9-9d42-9ba138d16abe\") " pod="opendatahub/test-curl-7548-139703280694576"
Jul 30 15:49:38.539860 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.539780    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280694576"
Jul 30 15:49:38.663762 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.663664    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7548-139703280694576"]
Jul 30 15:49:38.666304 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:49:38.666274    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fc04d30_2d55_41e9_9d42_9ba138d16abe.slice/crio-596d5dfa42ee1bb9ef9c36122b2eadb25ae7c52aa8fba41eb258d63af0074a96 WatchSource:0}: Error finding container 596d5dfa42ee1bb9ef9c36122b2eadb25ae7c52aa8fba41eb258d63af0074a96: Status 404 returned error can't find the container with id 596d5dfa42ee1bb9ef9c36122b2eadb25ae7c52aa8fba41eb258d63af0074a96
Jul 30 15:49:38.667724 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:38.667708    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 15:49:39.454979 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:39.454954    2582 generic.go:356] "Generic (PLEG): container finished" podID="9fc04d30-2d55-41e9-9d42-9ba138d16abe" containerID="e1bba592d2587c250312bc6aec2f42393fb0629da8b6ccea71f4ab8758515ae8" exitCode=0
Jul 30 15:49:39.455283 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:39.455042    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7548-139703280694576" event={"ID":"9fc04d30-2d55-41e9-9d42-9ba138d16abe","Type":"ContainerDied","Data":"e1bba592d2587c250312bc6aec2f42393fb0629da8b6ccea71f4ab8758515ae8"}
Jul 30 15:49:39.455283 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:39.455084    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7548-139703280694576" event={"ID":"9fc04d30-2d55-41e9-9d42-9ba138d16abe","Type":"ContainerStarted","Data":"596d5dfa42ee1bb9ef9c36122b2eadb25ae7c52aa8fba41eb258d63af0074a96"}
Jul 30 15:49:40.585986 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:40.585961    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280694576"
Jul 30 15:49:40.608459 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:40.608436    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7548-139703280694576_9fc04d30-2d55-41e9-9d42-9ba138d16abe/test-curl-7548-139703280694576/0.log"
Jul 30 15:49:40.636542 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:40.636516    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7548-139703280694576"]
Jul 30 15:49:40.640602 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:40.640581    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7548-139703280694576"]
Jul 30 15:49:40.645288 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:40.645269    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hd5h\" (UniqueName: \"kubernetes.io/projected/9fc04d30-2d55-41e9-9d42-9ba138d16abe-kube-api-access-8hd5h\") pod \"9fc04d30-2d55-41e9-9d42-9ba138d16abe\" (UID: \"9fc04d30-2d55-41e9-9d42-9ba138d16abe\") "
Jul 30 15:49:40.647492 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:40.647470    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fc04d30-2d55-41e9-9d42-9ba138d16abe-kube-api-access-8hd5h" (OuterVolumeSpecName: "kube-api-access-8hd5h") pod "9fc04d30-2d55-41e9-9d42-9ba138d16abe" (UID: "9fc04d30-2d55-41e9-9d42-9ba138d16abe"). InnerVolumeSpecName "kube-api-access-8hd5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:49:40.746021 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:40.745941    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hd5h\" (UniqueName: \"kubernetes.io/projected/9fc04d30-2d55-41e9-9d42-9ba138d16abe-kube-api-access-8hd5h\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:49:40.806570 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:40.806538    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fc04d30-2d55-41e9-9d42-9ba138d16abe" path="/var/lib/kubelet/pods/9fc04d30-2d55-41e9-9d42-9ba138d16abe/volumes"
Jul 30 15:49:41.464578 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:41.464547    2582 scope.go:117] "RemoveContainer" containerID="e1bba592d2587c250312bc6aec2f42393fb0629da8b6ccea71f4ab8758515ae8"
Jul 30 15:49:41.464578 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:41.464564    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280694576"
Jul 30 15:49:43.238549 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.238517    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7548-139703280203936"]
Jul 30 15:49:43.238993 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.238784    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fc04d30-2d55-41e9-9d42-9ba138d16abe" containerName="test-curl-7548-139703280694576"
Jul 30 15:49:43.238993 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.238796    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fc04d30-2d55-41e9-9d42-9ba138d16abe" containerName="test-curl-7548-139703280694576"
Jul 30 15:49:43.238993 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.238845    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fc04d30-2d55-41e9-9d42-9ba138d16abe" containerName="test-curl-7548-139703280694576"
Jul 30 15:49:43.246772 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.246740    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280203936"
Jul 30 15:49:43.246983 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.246940    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7548-139703280203936"]
Jul 30 15:49:43.249025 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.249004    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kpckp\""
Jul 30 15:49:43.365077 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.365037    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s4kj\" (UniqueName: \"kubernetes.io/projected/45c8e19c-6731-4cff-abbd-03f6d7338cf6-kube-api-access-7s4kj\") pod \"test-curl-7548-139703280203936\" (UID: \"45c8e19c-6731-4cff-abbd-03f6d7338cf6\") " pod="opendatahub/test-curl-7548-139703280203936"
Jul 30 15:49:43.466409 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.466362    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s4kj\" (UniqueName: \"kubernetes.io/projected/45c8e19c-6731-4cff-abbd-03f6d7338cf6-kube-api-access-7s4kj\") pod \"test-curl-7548-139703280203936\" (UID: \"45c8e19c-6731-4cff-abbd-03f6d7338cf6\") " pod="opendatahub/test-curl-7548-139703280203936"
Jul 30 15:49:43.475727 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.475704    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s4kj\" (UniqueName: \"kubernetes.io/projected/45c8e19c-6731-4cff-abbd-03f6d7338cf6-kube-api-access-7s4kj\") pod \"test-curl-7548-139703280203936\" (UID: \"45c8e19c-6731-4cff-abbd-03f6d7338cf6\") " pod="opendatahub/test-curl-7548-139703280203936"
Jul 30 15:49:43.556372 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.556315    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280203936"
Jul 30 15:49:43.683094 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:43.683071    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7548-139703280203936"]
Jul 30 15:49:43.685457 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:49:43.685430    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45c8e19c_6731_4cff_abbd_03f6d7338cf6.slice/crio-ae6c6197f33dc5d2fda85db27849eb5d3c06c57a63da457b1288ca53ab1fe630 WatchSource:0}: Error finding container ae6c6197f33dc5d2fda85db27849eb5d3c06c57a63da457b1288ca53ab1fe630: Status 404 returned error can't find the container with id ae6c6197f33dc5d2fda85db27849eb5d3c06c57a63da457b1288ca53ab1fe630
Jul 30 15:49:44.480472 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:44.480442    2582 generic.go:356] "Generic (PLEG): container finished" podID="45c8e19c-6731-4cff-abbd-03f6d7338cf6" containerID="8f381cfc923a44882946682c69b9d0f92f853e60cec6c8e965943a6012159810" exitCode=0
Jul 30 15:49:44.480972 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:44.480537    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7548-139703280203936" event={"ID":"45c8e19c-6731-4cff-abbd-03f6d7338cf6","Type":"ContainerDied","Data":"8f381cfc923a44882946682c69b9d0f92f853e60cec6c8e965943a6012159810"}
Jul 30 15:49:44.480972 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:44.480576    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7548-139703280203936" event={"ID":"45c8e19c-6731-4cff-abbd-03f6d7338cf6","Type":"ContainerStarted","Data":"ae6c6197f33dc5d2fda85db27849eb5d3c06c57a63da457b1288ca53ab1fe630"}
Jul 30 15:49:44.725270 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:44.725233    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:49:45.619536 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:45.619514    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280203936"
Jul 30 15:49:45.638947 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:45.638925    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7548-139703280203936_45c8e19c-6731-4cff-abbd-03f6d7338cf6/test-curl-7548-139703280203936/0.log"
Jul 30 15:49:45.667158 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:45.667132    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7548-139703280203936"]
Jul 30 15:49:45.672336 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:45.672317    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7548-139703280203936"]
Jul 30 15:49:45.682617 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:45.682599    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s4kj\" (UniqueName: \"kubernetes.io/projected/45c8e19c-6731-4cff-abbd-03f6d7338cf6-kube-api-access-7s4kj\") pod \"45c8e19c-6731-4cff-abbd-03f6d7338cf6\" (UID: \"45c8e19c-6731-4cff-abbd-03f6d7338cf6\") "
Jul 30 15:49:45.684689 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:45.684668    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45c8e19c-6731-4cff-abbd-03f6d7338cf6-kube-api-access-7s4kj" (OuterVolumeSpecName: "kube-api-access-7s4kj") pod "45c8e19c-6731-4cff-abbd-03f6d7338cf6" (UID: "45c8e19c-6731-4cff-abbd-03f6d7338cf6"). InnerVolumeSpecName "kube-api-access-7s4kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:49:45.783375 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:45.783307    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s4kj\" (UniqueName: \"kubernetes.io/projected/45c8e19c-6731-4cff-abbd-03f6d7338cf6-kube-api-access-7s4kj\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:49:46.489625 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:46.489601    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280203936"
Jul 30 15:49:46.489625 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:46.489629    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae6c6197f33dc5d2fda85db27849eb5d3c06c57a63da457b1288ca53ab1fe630"
Jul 30 15:49:46.498532 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:49:46.498508    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7548-139703280203936\" is forbidden: User \"system:node:ip-10-0-133-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-92.ec2.internal' and this object" podUID="45c8e19c-6731-4cff-abbd-03f6d7338cf6" pod="opendatahub/test-curl-7548-139703280203936"
Jul 30 15:49:46.806787 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:46.806714    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45c8e19c-6731-4cff-abbd-03f6d7338cf6" path="/var/lib/kubelet/pods/45c8e19c-6731-4cff-abbd-03f6d7338cf6/volumes"
Jul 30 15:49:47.918607 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:47.918546    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm" podUID="032dafa8-1b8a-4a28-bc38-b7b5808b1d77" containerName="main" containerID="cri-o://8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5" gracePeriod=45
Jul 30 15:49:48.176288 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.176230    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:49:48.278370 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.278339    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7548-139703280859152"]
Jul 30 15:49:48.278656 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.278644    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="032dafa8-1b8a-4a28-bc38-b7b5808b1d77" containerName="main"
Jul 30 15:49:48.278706 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.278658    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="032dafa8-1b8a-4a28-bc38-b7b5808b1d77" containerName="main"
Jul 30 15:49:48.278706 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.278666    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45c8e19c-6731-4cff-abbd-03f6d7338cf6" containerName="test-curl-7548-139703280203936"
Jul 30 15:49:48.278706 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.278673    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="45c8e19c-6731-4cff-abbd-03f6d7338cf6" containerName="test-curl-7548-139703280203936"
Jul 30 15:49:48.278799 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.278721    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="45c8e19c-6731-4cff-abbd-03f6d7338cf6" containerName="test-curl-7548-139703280203936"
Jul 30 15:49:48.278799 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.278729    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="032dafa8-1b8a-4a28-bc38-b7b5808b1d77" containerName="main"
Jul 30 15:49:48.281431 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.281416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280859152"
Jul 30 15:49:48.283294 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.283275    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kpckp\""
Jul 30 15:49:48.286017 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.285996    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7548-139703280859152"]
Jul 30 15:49:48.301575 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.301554    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkv4m\" (UniqueName: \"kubernetes.io/projected/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-kube-api-access-jkv4m\") pod \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") "
Jul 30 15:49:48.301669 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.301591    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tls-certs\") pod \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") "
Jul 30 15:49:48.301669 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.301653    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tmp-dir\") pod \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") "
Jul 30 15:49:48.301760 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.301684    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-model-cache\") pod \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") "
Jul 30 15:49:48.301760 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.301723    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-dshm\") pod \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") "
Jul 30 15:49:48.301865 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.301769    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-home\") pod \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\" (UID: \"032dafa8-1b8a-4a28-bc38-b7b5808b1d77\") "
Jul 30 15:49:48.301942 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.301916    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "032dafa8-1b8a-4a28-bc38-b7b5808b1d77" (UID: "032dafa8-1b8a-4a28-bc38-b7b5808b1d77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:49:48.302007 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.301932    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-model-cache" (OuterVolumeSpecName: "model-cache") pod "032dafa8-1b8a-4a28-bc38-b7b5808b1d77" (UID: "032dafa8-1b8a-4a28-bc38-b7b5808b1d77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:49:48.302072 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.302053    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tmp-dir\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:49:48.302072 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.302079    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-model-cache\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:49:48.302072 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.302046    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-home" (OuterVolumeSpecName: "home") pod "032dafa8-1b8a-4a28-bc38-b7b5808b1d77" (UID: "032dafa8-1b8a-4a28-bc38-b7b5808b1d77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:49:48.303989 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.303961    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-kube-api-access-jkv4m" (OuterVolumeSpecName: "kube-api-access-jkv4m") pod "032dafa8-1b8a-4a28-bc38-b7b5808b1d77" (UID: "032dafa8-1b8a-4a28-bc38-b7b5808b1d77"). InnerVolumeSpecName "kube-api-access-jkv4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:49:48.304085 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.303992    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-dshm" (OuterVolumeSpecName: "dshm") pod "032dafa8-1b8a-4a28-bc38-b7b5808b1d77" (UID: "032dafa8-1b8a-4a28-bc38-b7b5808b1d77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:49:48.304179 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.304159    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "032dafa8-1b8a-4a28-bc38-b7b5808b1d77" (UID: "032dafa8-1b8a-4a28-bc38-b7b5808b1d77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:49:48.402583 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.402546    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdfs9\" (UniqueName: \"kubernetes.io/projected/c0a5db45-65be-4958-98c6-b55bfc1b1a51-kube-api-access-hdfs9\") pod \"test-curl-7548-139703280859152\" (UID: \"c0a5db45-65be-4958-98c6-b55bfc1b1a51\") " pod="opendatahub/test-curl-7548-139703280859152"
Jul 30 15:49:48.402738 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.402658    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkv4m\" (UniqueName: \"kubernetes.io/projected/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-kube-api-access-jkv4m\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:49:48.402738 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.402676    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-tls-certs\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:49:48.402738 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.402689    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-dshm\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:49:48.402738 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.402700    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/032dafa8-1b8a-4a28-bc38-b7b5808b1d77-home\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:49:48.499882 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.499856    2582 generic.go:356] "Generic (PLEG): container finished" podID="032dafa8-1b8a-4a28-bc38-b7b5808b1d77" containerID="8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5" exitCode=0
Jul 30 15:49:48.500018 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.499900    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"
Jul 30 15:49:48.500018 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.499949    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm" event={"ID":"032dafa8-1b8a-4a28-bc38-b7b5808b1d77","Type":"ContainerDied","Data":"8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5"}
Jul 30 15:49:48.500018 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.499993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm" event={"ID":"032dafa8-1b8a-4a28-bc38-b7b5808b1d77","Type":"ContainerDied","Data":"16e4f6307097c6dcf586dfbd9a847625551beb420c2854c9559b543a58efdd96"}
Jul 30 15:49:48.500018 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.500016    2582 scope.go:117] "RemoveContainer" containerID="8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5"
Jul 30 15:49:48.503093 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.503069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdfs9\" (UniqueName: \"kubernetes.io/projected/c0a5db45-65be-4958-98c6-b55bfc1b1a51-kube-api-access-hdfs9\") pod \"test-curl-7548-139703280859152\" (UID: \"c0a5db45-65be-4958-98c6-b55bfc1b1a51\") " pod="opendatahub/test-curl-7548-139703280859152"
Jul 30 15:49:48.508998 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.508978    2582 scope.go:117] "RemoveContainer" containerID="8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5"
Jul 30 15:49:48.509282 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:49:48.509261    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5\": container with ID starting with 8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5 not found: ID does not exist" containerID="8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5"
Jul 30 15:49:48.509342 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.509291    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5"} err="failed to get container status \"8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5\": rpc error: code = NotFound desc = could not find container \"8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5\": container with ID starting with 8d6c75b9e325538cac3d7f1cc84caaad455a4e2c91be189abfdfd247e82151b5 not found: ID does not exist"
Jul 30 15:49:48.510630 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.510613    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdfs9\" (UniqueName: \"kubernetes.io/projected/c0a5db45-65be-4958-98c6-b55bfc1b1a51-kube-api-access-hdfs9\") pod \"test-curl-7548-139703280859152\" (UID: \"c0a5db45-65be-4958-98c6-b55bfc1b1a51\") " pod="opendatahub/test-curl-7548-139703280859152"
Jul 30 15:49:48.523370 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.523348    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"]
Jul 30 15:49:48.526892 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.526873    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-845a26/rate-test-model-845a26-kserve-795c45c4dc-s6kcm"]
Jul 30 15:49:48.591691 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.591669    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280859152"
Jul 30 15:49:48.719278 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.719250    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7548-139703280859152"]
Jul 30 15:49:48.721895 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:49:48.721869    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0a5db45_65be_4958_98c6_b55bfc1b1a51.slice/crio-51fe202855b93f2339595a1cadb42427793454fcece4e55b9a3bd82d37df50cb WatchSource:0}: Error finding container 51fe202855b93f2339595a1cadb42427793454fcece4e55b9a3bd82d37df50cb: Status 404 returned error can't find the container with id 51fe202855b93f2339595a1cadb42427793454fcece4e55b9a3bd82d37df50cb
Jul 30 15:49:48.806494 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:48.806469    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="032dafa8-1b8a-4a28-bc38-b7b5808b1d77" path="/var/lib/kubelet/pods/032dafa8-1b8a-4a28-bc38-b7b5808b1d77/volumes"
Jul 30 15:49:49.506861 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:49.506833    2582 generic.go:356] "Generic (PLEG): container finished" podID="c0a5db45-65be-4958-98c6-b55bfc1b1a51" containerID="de851d740785ec4c9f2b8acb9cc2b112daabe344bd8c1bb64d2b352e43a94eb6" exitCode=0
Jul 30 15:49:49.507241 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:49.506918    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7548-139703280859152" event={"ID":"c0a5db45-65be-4958-98c6-b55bfc1b1a51","Type":"ContainerDied","Data":"de851d740785ec4c9f2b8acb9cc2b112daabe344bd8c1bb64d2b352e43a94eb6"}
Jul 30 15:49:49.507241 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:49.506952    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7548-139703280859152" event={"ID":"c0a5db45-65be-4958-98c6-b55bfc1b1a51","Type":"ContainerStarted","Data":"51fe202855b93f2339595a1cadb42427793454fcece4e55b9a3bd82d37df50cb"}
Jul 30 15:49:50.641016 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:50.640992    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280859152"
Jul 30 15:49:50.661706 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:50.661684    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7548-139703280859152_c0a5db45-65be-4958-98c6-b55bfc1b1a51/test-curl-7548-139703280859152/0.log"
Jul 30 15:49:50.687258 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:50.687236    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7548-139703280859152"]
Jul 30 15:49:50.691968 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:50.691943    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7548-139703280859152"]
Jul 30 15:49:50.716905 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:50.716881    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdfs9\" (UniqueName: \"kubernetes.io/projected/c0a5db45-65be-4958-98c6-b55bfc1b1a51-kube-api-access-hdfs9\") pod \"c0a5db45-65be-4958-98c6-b55bfc1b1a51\" (UID: \"c0a5db45-65be-4958-98c6-b55bfc1b1a51\") "
Jul 30 15:49:50.718994 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:50.718972    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0a5db45-65be-4958-98c6-b55bfc1b1a51-kube-api-access-hdfs9" (OuterVolumeSpecName: "kube-api-access-hdfs9") pod "c0a5db45-65be-4958-98c6-b55bfc1b1a51" (UID: "c0a5db45-65be-4958-98c6-b55bfc1b1a51"). InnerVolumeSpecName "kube-api-access-hdfs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:49:50.807367 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:50.807294    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0a5db45-65be-4958-98c6-b55bfc1b1a51" path="/var/lib/kubelet/pods/c0a5db45-65be-4958-98c6-b55bfc1b1a51/volumes"
Jul 30 15:49:50.817461 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:50.817423    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdfs9\" (UniqueName: \"kubernetes.io/projected/c0a5db45-65be-4958-98c6-b55bfc1b1a51-kube-api-access-hdfs9\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:49:51.234840 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.234803    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:51.518018 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.517945    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7548-139703280859152"
Jul 30 15:49:51.518176 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.517952    2582 scope.go:117] "RemoveContainer" containerID="de851d740785ec4c9f2b8acb9cc2b112daabe344bd8c1bb64d2b352e43a94eb6"
Jul 30 15:49:51.536289 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.536266    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"]
Jul 30 15:49:51.536618 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.536605    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0a5db45-65be-4958-98c6-b55bfc1b1a51" containerName="test-curl-7548-139703280859152"
Jul 30 15:49:51.536666 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.536620    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0a5db45-65be-4958-98c6-b55bfc1b1a51" containerName="test-curl-7548-139703280859152"
Jul 30 15:49:51.536701 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.536666    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0a5db45-65be-4958-98c6-b55bfc1b1a51" containerName="test-curl-7548-139703280859152"
Jul 30 15:49:51.540701 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.540683    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.542729 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.542713    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-10fa8b-openshift-default-dockercfg-xgk4g\""
Jul 30 15:49:51.552144 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.552123    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"]
Jul 30 15:49:51.561942 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.561916    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:51.624960 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.624768    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-token\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.624960 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.624833    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ldkr\" (UniqueName: \"kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-kube-api-access-5ldkr\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.624960 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.624872    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-certs\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.624960 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.624905    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-data\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.624960 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.624962    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-podinfo\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.625354 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.624983    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istiod-ca-cert\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.625354 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.625009    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-credential-socket\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.625354 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.625070    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-socket\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.625354 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.625098    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-envoy\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.725736 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.725700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-podinfo\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.725736 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.725735    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istiod-ca-cert\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.725766    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-credential-socket\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.725813    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-socket\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.725840    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-envoy\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.725870    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-token\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.725903    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ldkr\" (UniqueName: \"kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-kube-api-access-5ldkr\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.725927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-certs\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.725955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-data\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726569 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.726341    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-socket\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726569 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.726418    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-data\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726569 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.726469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-certs\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726569 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.726522    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istiod-ca-cert\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.726569 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.726562    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-credential-socket\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.728273 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.728251    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-envoy\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.728357 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.728320    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-podinfo\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.735637 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.735619    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ldkr\" (UniqueName: \"kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-kube-api-access-5ldkr\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.735726 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.735686    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-token\") pod \"e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") " pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.829101 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.829004    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:51.850126 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.850103    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:51.985467 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:51.985437    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"]
Jul 30 15:49:51.988004 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:49:51.987981    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cfb7e6e_a088_4ce7_b24d_b6391da26150.slice/crio-264ea6510afd6226d7079ff374ba3fb596f9383e431c2fe94b8e0ed63075c319 WatchSource:0}: Error finding container 264ea6510afd6226d7079ff374ba3fb596f9383e431c2fe94b8e0ed63075c319: Status 404 returned error can't find the container with id 264ea6510afd6226d7079ff374ba3fb596f9383e431c2fe94b8e0ed63075c319
Jul 30 15:49:52.523531 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:52.523496    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" event={"ID":"8cfb7e6e-a088-4ce7-b24d-b6391da26150","Type":"ContainerStarted","Data":"264ea6510afd6226d7079ff374ba3fb596f9383e431c2fe94b8e0ed63075c319"}
Jul 30 15:49:55.109331 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:55.109301    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:49:55.109603 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:55.109365    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:49:55.109603 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:55.109415    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:49:55.539159 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:55.539120    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" event={"ID":"8cfb7e6e-a088-4ce7-b24d-b6391da26150","Type":"ContainerStarted","Data":"921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b"}
Jul 30 15:49:55.563089 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:55.563043    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" podStartSLOduration=1.444315676 podStartE2EDuration="4.563029127s" totalImagesPullingTime="3.118713451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:49:55.562293627 +0000 UTC m=+3701.235739739" watchObservedRunningTime="2026-07-30 15:49:55.563029127 +0000 UTC m=+3701.236475239"
Jul 30 15:49:55.851247 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:55.851162    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:56.856077 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:56.856047    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:57.547546 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:57.547504    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:57.548382 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:57.548359    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:49:57.603963 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:57.603929    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:57.699881 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:57.699849    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:58.696480 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.696450    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"]
Jul 30 15:49:58.699598 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.699581    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:58.701585 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.701557    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-10fa8b-serving-cert\""
Jul 30 15:49:58.709518 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.709488    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"]
Jul 30 15:49:58.726177 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.726155    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:58.784006 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.783979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:58.784121 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.784009    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:58.784121 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.784028    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw8sx\" (UniqueName: \"kubernetes.io/projected/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-kube-api-access-bw8sx\") pod \"maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:58.885035 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.885011    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:58.885172 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.885041    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:58.885172 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.885063    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw8sx\" (UniqueName: \"kubernetes.io/projected/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-kube-api-access-bw8sx\") pod \"maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:58.885795 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.885768    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:58.887680 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.887656    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:58.892320 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.892297    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw8sx\" (UniqueName: \"kubernetes.io/projected/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-kube-api-access-bw8sx\") pod \"maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:58.938088 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:58.938058    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:59.010336 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:59.010270    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:49:59.151517 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:59.151317    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"]
Jul 30 15:49:59.153902 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:49:59.153872    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8d1fc1_e4a6_4454_bc99_c6e17392349f.slice/crio-aa022c66072f220fa8ff9c796775c0574dddf05015963a2597c0d23f252d2f48 WatchSource:0}: Error finding container aa022c66072f220fa8ff9c796775c0574dddf05015963a2597c0d23f252d2f48: Status 404 returned error can't find the container with id aa022c66072f220fa8ff9c796775c0574dddf05015963a2597c0d23f252d2f48
Jul 30 15:49:59.364221 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:59.364138    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:59.459978 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:59.459945    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:49:59.559770 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:49:59.559737    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z" event={"ID":"cc8d1fc1-e4a6-4454-bc99-c6e17392349f","Type":"ContainerStarted","Data":"aa022c66072f220fa8ff9c796775c0574dddf05015963a2597c0d23f252d2f48"}
Jul 30 15:50:00.568045 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:00.568011    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z" event={"ID":"cc8d1fc1-e4a6-4454-bc99-c6e17392349f","Type":"ContainerStarted","Data":"124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa"}
Jul 30 15:50:00.568472 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:00.568133    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:50:00.583937 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:00.583894    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z" podStartSLOduration=1.9627610199999999 podStartE2EDuration="2.583879197s" totalImagesPullingTime="621.118177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:50:00.582334849 +0000 UTC m=+3706.255780962" watchObservedRunningTime="2026-07-30 15:50:00.583879197 +0000 UTC m=+3706.257325309"
Jul 30 15:50:05.231166 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:05.231124    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:50:06.577109 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:06.577080    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:50:13.589600 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:13.589554    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 30 15:50:13.589993 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:13.589615    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 15:50:14.630484 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:14.630359    2582 generic.go:356] "Generic (PLEG): container finished" podID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerID="921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b" exitCode=0
Jul 30 15:50:14.630484 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:14.630442    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" event={"ID":"8cfb7e6e-a088-4ce7-b24d-b6391da26150","Type":"ContainerDied","Data":"921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b"}
Jul 30 15:50:14.630937 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:14.630807    2582 scope.go:117] "RemoveContainer" containerID="921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b"
Jul 30 15:50:14.632716 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:14.632683    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:50:14.632834 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:14.632755    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:50:14.632834 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:14.632801    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:50:15.636788 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:15.636755    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" event={"ID":"8cfb7e6e-a088-4ce7-b24d-b6391da26150","Type":"ContainerStarted","Data":"41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac"}
Jul 30 15:50:15.850434 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:15.850378    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:50:15.850866 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:15.850837    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:50:15.850964 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:15.850909    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 30 15:50:16.850897 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:16.850855    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:50:16.851387 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:16.850923    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 30 15:50:17.850782 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:17.850737    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:50:17.850997 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:17.850819    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 30 15:50:18.850675 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:18.850641    2582 patch_prober.go:28] interesting pod/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 30 15:50:18.850991 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:18.850953    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 30 15:50:19.854079 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:19.854046    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:50:19.854502 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:19.854318    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:50:19.855050 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:19.855036    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:50:22.713434 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:22.713374    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:23.475887 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.475855    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:23.787012 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.786934    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"]
Jul 30 15:50:23.792094 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.792071    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.794206 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.794187    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-0d4073-openshift-default-dockercfg-q4h55\""
Jul 30 15:50:23.802420 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.802384    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"]
Jul 30 15:50:23.818793 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.818770    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:23.893759 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.893727    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-certs\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.893930 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.893766    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istiod-ca-cert\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.893930 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.893789    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-token\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.893930 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.893814    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-data\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.893930 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.893873    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-socket\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.893930 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.893890    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmkxz\" (UniqueName: \"kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-kube-api-access-rmkxz\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.894143 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.893932    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-podinfo\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.894143 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.893976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-credential-socket\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.894143 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.894021    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-envoy\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995016 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.994978    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-podinfo\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995193 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995031    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-credential-socket\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995193 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-envoy\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995193 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995124    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-certs\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995336 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995243    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istiod-ca-cert\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995336 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995282    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-token\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995336 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995314    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-data\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995519 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995353    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-socket\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995519 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995377    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmkxz\" (UniqueName: \"kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-kube-api-access-rmkxz\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995628 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995589    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-credential-socket\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995684 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995639    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-certs\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.995785 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.995764    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-socket\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.996024 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.996003    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istiod-ca-cert\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.996087 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.996009    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-data\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.997502 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.997484    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-envoy\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:23.997712 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:23.997696    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-podinfo\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:24.009086 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.009060    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-token\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:24.010491 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.010469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmkxz\" (UniqueName: \"kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-kube-api-access-rmkxz\") pod \"e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") " pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:24.102030 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.101948    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:24.237035 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.236864    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"]
Jul 30 15:50:24.239191 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:50:24.239158    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a17dbe_4b22_478c_bc66_1d229efdb6fb.slice/crio-5d035add9b6d3e51e084d32561ffc8592c4e333d8fdc5540eaf348fae43d6459 WatchSource:0}: Error finding container 5d035add9b6d3e51e084d32561ffc8592c4e333d8fdc5540eaf348fae43d6459: Status 404 returned error can't find the container with id 5d035add9b6d3e51e084d32561ffc8592c4e333d8fdc5540eaf348fae43d6459
Jul 30 15:50:24.239304 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.239275    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:24.241312 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.241284    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:50:24.241377 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.241343    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:50:24.241377 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.241369    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:50:24.686949 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.686914    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" event={"ID":"04a17dbe-4b22-478c-bc66-1d229efdb6fb","Type":"ContainerStarted","Data":"29bc31b6c898c58232ca9ec1c575adff624c1cb929e9cd0fd75731d7532ee761"}
Jul 30 15:50:24.687116 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.686993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" event={"ID":"04a17dbe-4b22-478c-bc66-1d229efdb6fb","Type":"ContainerStarted","Data":"5d035add9b6d3e51e084d32561ffc8592c4e333d8fdc5540eaf348fae43d6459"}
Jul 30 15:50:24.706872 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:24.706824    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" podStartSLOduration=1.706810109 podStartE2EDuration="1.706810109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:50:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:50:24.705443783 +0000 UTC m=+3730.378889893" watchObservedRunningTime="2026-07-30 15:50:24.706810109 +0000 UTC m=+3730.380256222"
Jul 30 15:50:25.103158 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:25.103072    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:25.108091 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:25.108066    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:25.691473 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:25.691441    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:25.692363 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:25.692348    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:26.670947 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:26.670913    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:26.770520 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:26.770484    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:27.757224 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:27.757192    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"]
Jul 30 15:50:27.765795 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:27.765769    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:27.767748 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:27.767726    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-0d4073-serving-cert\""
Jul 30 15:50:27.772304 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:27.772274    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"]
Jul 30 15:50:27.806044 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:27.806015    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:27.931439 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:27.931407    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24220e-995e-4476-855c-aee920b4709e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:27.931628 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:27.931449    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a24220e-995e-4476-855c-aee920b4709e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:27.931628 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:27.931484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-626dg\" (UniqueName: \"kubernetes.io/projected/4a24220e-995e-4476-855c-aee920b4709e-kube-api-access-626dg\") pod \"maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:28.032844 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.032758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-626dg\" (UniqueName: \"kubernetes.io/projected/4a24220e-995e-4476-855c-aee920b4709e-kube-api-access-626dg\") pod \"maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:28.032844 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.032824    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24220e-995e-4476-855c-aee920b4709e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:28.033079 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.032849    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a24220e-995e-4476-855c-aee920b4709e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:28.033079 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:50:28.032941    2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-serving-cert: secret "maas-api-e2e-ipp-b-0d4073-serving-cert" not found
Jul 30 15:50:28.033079 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:50:28.033000    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a24220e-995e-4476-855c-aee920b4709e-maas-api-tls podName:4a24220e-995e-4476-855c-aee920b4709e nodeName:}" failed. No retries permitted until 2026-07-30 15:50:28.532985698 +0000 UTC m=+3734.206431790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4a24220e-995e-4476-855c-aee920b4709e-maas-api-tls") pod "maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl" (UID: "4a24220e-995e-4476-855c-aee920b4709e") : secret "maas-api-e2e-ipp-b-0d4073-serving-cert" not found
Jul 30 15:50:28.033478 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.033460    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24220e-995e-4476-855c-aee920b4709e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:28.041581 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.041559    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-626dg\" (UniqueName: \"kubernetes.io/projected/4a24220e-995e-4476-855c-aee920b4709e-kube-api-access-626dg\") pod \"maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:28.177016 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.176978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:28.411970 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.411871    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:28.536664 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.536633    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a24220e-995e-4476-855c-aee920b4709e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:28.539433 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.539383    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a24220e-995e-4476-855c-aee920b4709e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:28.677523 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.677432    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:28.811763 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:28.811738    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"]
Jul 30 15:50:28.814523 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:50:28.814500    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a24220e_995e_4476_855c_aee920b4709e.slice/crio-cd7e67e017b6a6264b5f4a306670f2b690eaeffbc3043232aefb6834bb113b6a WatchSource:0}: Error finding container cd7e67e017b6a6264b5f4a306670f2b690eaeffbc3043232aefb6834bb113b6a: Status 404 returned error can't find the container with id cd7e67e017b6a6264b5f4a306670f2b690eaeffbc3043232aefb6834bb113b6a
Jul 30 15:50:29.303376 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:29.303340    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:29.709500 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:29.709462    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl" event={"ID":"4a24220e-995e-4476-855c-aee920b4709e","Type":"ContainerStarted","Data":"7674a13bcc8032f694f1c4b8283ea554cc48f98d452468284fac16e265b76795"}
Jul 30 15:50:29.709694 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:29.709512    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl" event={"ID":"4a24220e-995e-4476-855c-aee920b4709e","Type":"ContainerStarted","Data":"cd7e67e017b6a6264b5f4a306670f2b690eaeffbc3043232aefb6834bb113b6a"}
Jul 30 15:50:29.709694 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:29.709573    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:29.727489 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:29.727445    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl" podStartSLOduration=2.376141004 podStartE2EDuration="2.727431287s" totalImagesPullingTime="351.290283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:50:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:50:29.725384325 +0000 UTC m=+3735.398830437" watchObservedRunningTime="2026-07-30 15:50:29.727431287 +0000 UTC m=+3735.400877399"
Jul 30 15:50:35.526133 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:35.526100    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:50:35.719575 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:35.719548    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:50:41.032432 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:41.032379    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:50:44.786878 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:44.786840    2582 generic.go:356] "Generic (PLEG): container finished" podID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerID="29bc31b6c898c58232ca9ec1c575adff624c1cb929e9cd0fd75731d7532ee761" exitCode=0
Jul 30 15:50:44.787305 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:44.786919    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" event={"ID":"04a17dbe-4b22-478c-bc66-1d229efdb6fb","Type":"ContainerDied","Data":"29bc31b6c898c58232ca9ec1c575adff624c1cb929e9cd0fd75731d7532ee761"}
Jul 30 15:50:44.787305 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:44.787300    2582 scope.go:117] "RemoveContainer" containerID="29bc31b6c898c58232ca9ec1c575adff624c1cb929e9cd0fd75731d7532ee761"
Jul 30 15:50:44.788263 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:44.788238    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:50:44.788315 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:44.788303    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:50:44.788352 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:44.788343    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:50:45.793794 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:45.793761    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" event={"ID":"04a17dbe-4b22-478c-bc66-1d229efdb6fb","Type":"ContainerStarted","Data":"41d587c418e02aaace44100d9dd09d346da4821b0735f087c8777e5b4d254221"}
Jul 30 15:50:46.102805 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:46.102704    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:46.102996 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:46.102970    2582 patch_prober.go:28] interesting pod/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 30 15:50:46.103075 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:46.103041    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 30 15:50:47.102942 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:47.102902    2582 patch_prober.go:28] interesting pod/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 30 15:50:47.103543 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:47.102965    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 30 15:50:48.102755 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:48.102718    2582 patch_prober.go:28] interesting pod/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 30 15:50:48.102935 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:48.102776    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 30 15:50:49.102532 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:49.102484    2582 patch_prober.go:28] interesting pod/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 30 15:50:49.103022 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:49.102552    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 30 15:50:50.106637 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:50.106606    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:50.107016 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:50.106867    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:50.107654 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:50.107633    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:50:51.745215 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:50:51.745179    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:51:18.949383 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:51:18.949306    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:51:22.518905 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:51:22.518859    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:51:46.476992 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:51:46.476946    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:51:54.622677 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:51:54.622642    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:52:05.153409 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.153364    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:05.571009 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.570976    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"]
Jul 30 15:52:05.574845 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.574822    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.577528 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.577502    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-2d48ec-openshift-default-dockercfg-fpvpm\""
Jul 30 15:52:05.597882 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.597856    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"]
Jul 30 15:52:05.639328 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.639295    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-credential-socket\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.639491 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.639354    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-podinfo\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.639491 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.639413    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-data\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.639626 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.639535    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-envoy\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.639626 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.639603    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-certs\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.639730 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.639627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eea024b-6be6-40d3-b693-b483a7da19f7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.639730 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.639708    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-token\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.639886 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.639754    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-socket\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.639886 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.639789    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbvmd\" (UniqueName: \"kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-kube-api-access-dbvmd\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741053 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741021    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-token\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741053 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-socket\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741320 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbvmd\" (UniqueName: \"kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-kube-api-access-dbvmd\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741320 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741114    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-credential-socket\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741320 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741164    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-podinfo\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741320 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741204    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-data\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741320 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741264    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-envoy\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741320 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-certs\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741668 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741434    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eea024b-6be6-40d3-b693-b483a7da19f7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741668 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741508    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-socket\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741668 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741568    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-credential-socket\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741668 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741597    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-certs\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.741819 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.741675    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-data\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.742119 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.742102    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eea024b-6be6-40d3-b693-b483a7da19f7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.743678 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.743656    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-envoy\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.743792 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.743782    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-podinfo\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.757298 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.757275    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-token\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.757670 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.757648    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbvmd\" (UniqueName: \"kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-kube-api-access-dbvmd\") pod \"e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") " pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:05.887014 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:05.886926    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:06.030964 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:06.030931    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"]
Jul 30 15:52:06.034078 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:52:06.034049    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eea024b_6be6_40d3_b693_b483a7da19f7.slice/crio-6110bac7d1d41286fa12992d13f68fe2649f0d006be4ae3f0122e102a18fa75b WatchSource:0}: Error finding container 6110bac7d1d41286fa12992d13f68fe2649f0d006be4ae3f0122e102a18fa75b: Status 404 returned error can't find the container with id 6110bac7d1d41286fa12992d13f68fe2649f0d006be4ae3f0122e102a18fa75b
Jul 30 15:52:06.036171 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:06.036141    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:52:06.036227 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:06.036202    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:52:06.036271 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:06.036229    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:52:06.139132 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:06.139040    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" event={"ID":"4eea024b-6be6-40d3-b693-b483a7da19f7","Type":"ContainerStarted","Data":"9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d"}
Jul 30 15:52:06.139132 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:06.139099    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" event={"ID":"4eea024b-6be6-40d3-b693-b483a7da19f7","Type":"ContainerStarted","Data":"6110bac7d1d41286fa12992d13f68fe2649f0d006be4ae3f0122e102a18fa75b"}
Jul 30 15:52:06.159889 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:06.159834    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" podStartSLOduration=1.159819116 podStartE2EDuration="1.159819116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 15:52:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:52:06.158204887 +0000 UTC m=+3831.831651013" watchObservedRunningTime="2026-07-30 15:52:06.159819116 +0000 UTC m=+3831.833265284"
Jul 30 15:52:06.887655 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:06.887621    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:06.908058 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:06.908031    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:07.893055 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:07.893016    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:08.135576 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:08.135549    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:08.147872 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:08.147798    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:08.148695 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:08.148676    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:08.238045 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:08.238011    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:09.279994 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:09.279964    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:09.488128 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:09.488099    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:09.794656 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:09.794623    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:09.898758 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:09.898712    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:14.258809 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:14.258773    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:52:22.214493 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:22.214454    2582 generic.go:356] "Generic (PLEG): container finished" podID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerID="9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d" exitCode=0
Jul 30 15:52:22.215006 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:22.214531    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" event={"ID":"4eea024b-6be6-40d3-b693-b483a7da19f7","Type":"ContainerDied","Data":"9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d"}
Jul 30 15:52:22.215075 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:22.215026    2582 scope.go:117] "RemoveContainer" containerID="9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d"
Jul 30 15:52:22.216138 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:22.216113    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:52:22.216210 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:22.216174    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:52:22.216210 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:22.216205    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 30 15:52:23.148285 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:23.148236    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:23.220957 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:23.220925    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" event={"ID":"4eea024b-6be6-40d3-b693-b483a7da19f7","Type":"ContainerStarted","Data":"e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3"}
Jul 30 15:52:23.888028 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:23.887986    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:23.889036 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:23.889013    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 15:52:23.889101 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:23.889064    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 15:52:24.888084 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:24.888046    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 15:52:24.888511 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:24.888101    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 15:52:25.887752 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:25.887717    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 15:52:25.888005 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:25.887784    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 15:52:26.887873 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:26.887840    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 30 15:52:26.888253 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:26.887903    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 30 15:52:27.891297 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:27.891268    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:27.891719 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:27.891604    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:27.892110 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:27.892092    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:33.226772 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:33.226739    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:33.943191 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:33.943157    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"]
Jul 30 15:52:33.946477 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:33.946461    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"
Jul 30 15:52:33.948655 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:33.948635    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:52:33.954611 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:33.954587    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"]
Jul 30 15:52:34.002693 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:34.002666    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72054363-69be-489a-b25c-6c671fc73a2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf\" (UID: \"72054363-69be-489a-b25c-6c671fc73a2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"
Jul 30 15:52:34.103707 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:34.103673    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72054363-69be-489a-b25c-6c671fc73a2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf\" (UID: \"72054363-69be-489a-b25c-6c671fc73a2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"
Jul 30 15:52:34.104271 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:34.104252    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72054363-69be-489a-b25c-6c671fc73a2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf\" (UID: \"72054363-69be-489a-b25c-6c671fc73a2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"
Jul 30 15:52:34.258668 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:34.258579    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"
Jul 30 15:52:34.386010 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:34.385977    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"]
Jul 30 15:52:34.388498 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:52:34.388466    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72054363_69be_489a_b25c_6c671fc73a2f.slice/crio-d0ddd869d9878538bd14440ffded6c0bf2d6cbbad3fc70e447d8be82522677ff WatchSource:0}: Error finding container d0ddd869d9878538bd14440ffded6c0bf2d6cbbad3fc70e447d8be82522677ff: Status 404 returned error can't find the container with id d0ddd869d9878538bd14440ffded6c0bf2d6cbbad3fc70e447d8be82522677ff
Jul 30 15:52:35.278403 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:35.278302    2582 generic.go:356] "Generic (PLEG): container finished" podID="72054363-69be-489a-b25c-6c671fc73a2f" containerID="bd3c450c8a65196dde9923e590a85eff1f4abfdd5ffbf3ad20f61a19acf129cd" exitCode=0
Jul 30 15:52:35.278749 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:35.278385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf" event={"ID":"72054363-69be-489a-b25c-6c671fc73a2f","Type":"ContainerDied","Data":"bd3c450c8a65196dde9923e590a85eff1f4abfdd5ffbf3ad20f61a19acf129cd"}
Jul 30 15:52:35.278749 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:35.278476    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf" event={"ID":"72054363-69be-489a-b25c-6c671fc73a2f","Type":"ContainerStarted","Data":"d0ddd869d9878538bd14440ffded6c0bf2d6cbbad3fc70e447d8be82522677ff"}
Jul 30 15:52:36.413152 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:36.413130    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"
Jul 30 15:52:36.420908 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:36.420888    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72054363-69be-489a-b25c-6c671fc73a2f-maas-api-service-ca\") pod \"72054363-69be-489a-b25c-6c671fc73a2f\" (UID: \"72054363-69be-489a-b25c-6c671fc73a2f\") "
Jul 30 15:52:36.421197 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:36.421176    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72054363-69be-489a-b25c-6c671fc73a2f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72054363-69be-489a-b25c-6c671fc73a2f" (UID: "72054363-69be-489a-b25c-6c671fc73a2f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:52:36.522224 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:36.522192    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72054363-69be-489a-b25c-6c671fc73a2f-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:37.289145 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:37.289110    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"
Jul 30 15:52:37.289319 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:37.289145    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf" event={"ID":"72054363-69be-489a-b25c-6c671fc73a2f","Type":"ContainerDied","Data":"d0ddd869d9878538bd14440ffded6c0bf2d6cbbad3fc70e447d8be82522677ff"}
Jul 30 15:52:37.289319 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:37.289193    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0ddd869d9878538bd14440ffded6c0bf2d6cbbad3fc70e447d8be82522677ff"
Jul 30 15:52:39.790432 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:39.790377    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:39.856917 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:39.856871    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:39.939416 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:39.939365    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:39.960567 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:39.960544    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"]
Jul 30 15:52:39.962770 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:39.962750    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2d48ec-pflwf"]
Jul 30 15:52:39.995568 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:39.995325    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:40.385317 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:40.385287    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:40.806973 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:40.806944    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72054363-69be-489a-b25c-6c671fc73a2f" path="/var/lib/kubelet/pods/72054363-69be-489a-b25c-6c671fc73a2f/volumes"
Jul 30 15:52:44.335582 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:44.335550    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:52:46.639686 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:46.639647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:46.740128 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:46.740094    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:46.909260 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:46.909184    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"]
Jul 30 15:52:46.909452 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:46.909421    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy" containerID="cri-o://e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3" gracePeriod=30
Jul 30 15:52:46.921913 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:46.921878    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:47.182490 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:47.182461    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:52:50.882266 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:50.882227    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"]
Jul 30 15:52:50.882676 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:50.882549    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72054363-69be-489a-b25c-6c671fc73a2f" containerName="revoke-keys"
Jul 30 15:52:50.882676 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:50.882560    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="72054363-69be-489a-b25c-6c671fc73a2f" containerName="revoke-keys"
Jul 30 15:52:50.882676 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:50.882618    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="72054363-69be-489a-b25c-6c671fc73a2f" containerName="revoke-keys"
Jul 30 15:52:50.886735 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:50.886717    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"
Jul 30 15:52:50.888692 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:50.888673    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:52:50.893702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:50.893676    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"]
Jul 30 15:52:50.944145 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:50.944112    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95\" (UID: \"3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"
Jul 30 15:52:51.045050 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:51.045022    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95\" (UID: \"3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"
Jul 30 15:52:51.045668 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:51.045649    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95\" (UID: \"3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"
Jul 30 15:52:51.197362 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:51.197338    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"
Jul 30 15:52:51.327445 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:51.327250    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"]
Jul 30 15:52:51.357671 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:51.357648    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95" event={"ID":"3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998","Type":"ContainerStarted","Data":"4e3f66444f15977811b42642ed600ab127a89462519dc678b27d37e0e3ca5828"}
Jul 30 15:52:52.149473 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.149447    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:52.253295 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253273    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-token\") pod \"4eea024b-6be6-40d3-b693-b483a7da19f7\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") "
Jul 30 15:52:52.253441 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253311    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbvmd\" (UniqueName: \"kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-kube-api-access-dbvmd\") pod \"4eea024b-6be6-40d3-b693-b483a7da19f7\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") "
Jul 30 15:52:52.253441 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253340    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eea024b-6be6-40d3-b693-b483a7da19f7-istiod-ca-cert\") pod \"4eea024b-6be6-40d3-b693-b483a7da19f7\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") "
Jul 30 15:52:52.253563 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253482    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-certs\") pod \"4eea024b-6be6-40d3-b693-b483a7da19f7\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") "
Jul 30 15:52:52.253563 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253512    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-data\") pod \"4eea024b-6be6-40d3-b693-b483a7da19f7\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") "
Jul 30 15:52:52.253563 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253546    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-podinfo\") pod \"4eea024b-6be6-40d3-b693-b483a7da19f7\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") "
Jul 30 15:52:52.253718 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253567    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-credential-socket\") pod \"4eea024b-6be6-40d3-b693-b483a7da19f7\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") "
Jul 30 15:52:52.253718 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253594    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-socket\") pod \"4eea024b-6be6-40d3-b693-b483a7da19f7\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") "
Jul 30 15:52:52.253718 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253636    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-envoy\") pod \"4eea024b-6be6-40d3-b693-b483a7da19f7\" (UID: \"4eea024b-6be6-40d3-b693-b483a7da19f7\") "
Jul 30 15:52:52.253718 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253688    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4eea024b-6be6-40d3-b693-b483a7da19f7" (UID: "4eea024b-6be6-40d3-b693-b483a7da19f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:52:52.253926 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253718    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eea024b-6be6-40d3-b693-b483a7da19f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4eea024b-6be6-40d3-b693-b483a7da19f7" (UID: "4eea024b-6be6-40d3-b693-b483a7da19f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:52:52.253926 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253842    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4eea024b-6be6-40d3-b693-b483a7da19f7" (UID: "4eea024b-6be6-40d3-b693-b483a7da19f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:52:52.254040 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253936    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eea024b-6be6-40d3-b693-b483a7da19f7-istiod-ca-cert\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:52.254040 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253953    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-certs\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:52.254040 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.253965    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-credential-socket\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:52.254040 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.254002    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4eea024b-6be6-40d3-b693-b483a7da19f7" (UID: "4eea024b-6be6-40d3-b693-b483a7da19f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:52:52.254728 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.254695    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "4eea024b-6be6-40d3-b693-b483a7da19f7" (UID: "4eea024b-6be6-40d3-b693-b483a7da19f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:52:52.255835 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.255800    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "4eea024b-6be6-40d3-b693-b483a7da19f7" (UID: "4eea024b-6be6-40d3-b693-b483a7da19f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:52:52.255951 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.255862    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4eea024b-6be6-40d3-b693-b483a7da19f7" (UID: "4eea024b-6be6-40d3-b693-b483a7da19f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:52:52.255951 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.255901    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-kube-api-access-dbvmd" (OuterVolumeSpecName: "kube-api-access-dbvmd") pod "4eea024b-6be6-40d3-b693-b483a7da19f7" (UID: "4eea024b-6be6-40d3-b693-b483a7da19f7"). InnerVolumeSpecName "kube-api-access-dbvmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:52:52.255951 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.255926    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4eea024b-6be6-40d3-b693-b483a7da19f7" (UID: "4eea024b-6be6-40d3-b693-b483a7da19f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:52:52.354582 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.354553    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-data\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:52.354582 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.354578    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-podinfo\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:52.354721 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.354591    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-workload-socket\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:52.354721 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.354602    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-envoy\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:52.354721 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.354613    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-istio-token\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:52.354721 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.354628    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbvmd\" (UniqueName: \"kubernetes.io/projected/4eea024b-6be6-40d3-b693-b483a7da19f7-kube-api-access-dbvmd\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:52.364049 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.364023    2582 generic.go:356] "Generic (PLEG): container finished" podID="3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998" containerID="c6926e4b30aa31dc463c557f5e14671977c4efb063bb405cde6e106a6375c798" exitCode=0
Jul 30 15:52:52.364159 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.364077    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95" event={"ID":"3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998","Type":"ContainerDied","Data":"c6926e4b30aa31dc463c557f5e14671977c4efb063bb405cde6e106a6375c798"}
Jul 30 15:52:52.365776 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.365747    2582 generic.go:356] "Generic (PLEG): container finished" podID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerID="e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3" exitCode=0
Jul 30 15:52:52.365863 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.365788    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"
Jul 30 15:52:52.365863 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.365822    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" event={"ID":"4eea024b-6be6-40d3-b693-b483a7da19f7","Type":"ContainerDied","Data":"e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3"}
Jul 30 15:52:52.365863 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.365858    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl" event={"ID":"4eea024b-6be6-40d3-b693-b483a7da19f7","Type":"ContainerDied","Data":"6110bac7d1d41286fa12992d13f68fe2649f0d006be4ae3f0122e102a18fa75b"}
Jul 30 15:52:52.365970 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.365879    2582 scope.go:117] "RemoveContainer" containerID="e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3"
Jul 30 15:52:52.375080 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.375065    2582 scope.go:117] "RemoveContainer" containerID="9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d"
Jul 30 15:52:52.383614 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.383599    2582 scope.go:117] "RemoveContainer" containerID="e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3"
Jul 30 15:52:52.383852 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:52:52.383832    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3\": container with ID starting with e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3 not found: ID does not exist" containerID="e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3"
Jul 30 15:52:52.383913 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.383858    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3"} err="failed to get container status \"e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3\": rpc error: code = NotFound desc = could not find container \"e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3\": container with ID starting with e13982225b3840ff8f5222637be8b4c2b7d427ce9609e8823bc1abdecf01e3e3 not found: ID does not exist"
Jul 30 15:52:52.383913 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.383873    2582 scope.go:117] "RemoveContainer" containerID="9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d"
Jul 30 15:52:52.384093 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:52:52.384077    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d\": container with ID starting with 9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d not found: ID does not exist" containerID="9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d"
Jul 30 15:52:52.384139 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.384097    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d"} err="failed to get container status \"9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d\": rpc error: code = NotFound desc = could not find container \"9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d\": container with ID starting with 9b357579068d57308dad0c93238ce0f93bcb21da5f38918f2536214cc13c8d8d not found: ID does not exist"
Jul 30 15:52:52.393153 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.393132    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"]
Jul 30 15:52:52.400648 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.400629    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2d48ec-openshift-default-6c4b6fb56-6sncl"]
Jul 30 15:52:52.807785 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:52.807755    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" path="/var/lib/kubelet/pods/4eea024b-6be6-40d3-b693-b483a7da19f7/volumes"
Jul 30 15:52:53.500698 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:53.500674    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"
Jul 30 15:52:53.564438 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:53.564389    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998-maas-api-service-ca\") pod \"3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998\" (UID: \"3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998\") "
Jul 30 15:52:53.564697 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:53.564678    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998" (UID: "3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:52:53.664980 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:53.664913    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:52:54.375709 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:54.375673    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95" event={"ID":"3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998","Type":"ContainerDied","Data":"4e3f66444f15977811b42642ed600ab127a89462519dc678b27d37e0e3ca5828"}
Jul 30 15:52:54.375709 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:54.375693    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"
Jul 30 15:52:54.375913 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:52:54.375731    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e3f66444f15977811b42642ed600ab127a89462519dc678b27d37e0e3ca5828"
Jul 30 15:53:06.386070 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.386026    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"]
Jul 30 15:53:06.386738 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.386360    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z" podUID="cc8d1fc1-e4a6-4454-bc99-c6e17392349f" containerName="maas-api" containerID="cri-o://124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa" gracePeriod=30
Jul 30 15:53:06.415948 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.415911    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:06.494627 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.494589    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:06.552533 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.552497    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:06.601319 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.601291    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"]
Jul 30 15:53:06.602807 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.602788    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10fa8b-cdc95"]
Jul 30 15:53:06.645202 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.645136    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:53:06.652900 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.652877    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:06.776177 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.776143    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw8sx\" (UniqueName: \"kubernetes.io/projected/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-kube-api-access-bw8sx\") pod \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") "
Jul 30 15:53:06.776361 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.776197    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-maas-api-tls\") pod \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") "
Jul 30 15:53:06.776361 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.776239    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-openshift-service-ca\") pod \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\" (UID: \"cc8d1fc1-e4a6-4454-bc99-c6e17392349f\") "
Jul 30 15:53:06.776695 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.776666    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc8d1fc1-e4a6-4454-bc99-c6e17392349f" (UID: "cc8d1fc1-e4a6-4454-bc99-c6e17392349f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:53:06.778425 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.778385    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc8d1fc1-e4a6-4454-bc99-c6e17392349f" (UID: "cc8d1fc1-e4a6-4454-bc99-c6e17392349f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:53:06.778501 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.778422    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-kube-api-access-bw8sx" (OuterVolumeSpecName: "kube-api-access-bw8sx") pod "cc8d1fc1-e4a6-4454-bc99-c6e17392349f" (UID: "cc8d1fc1-e4a6-4454-bc99-c6e17392349f"). InnerVolumeSpecName "kube-api-access-bw8sx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:53:06.807220 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.807190    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998" path="/var/lib/kubelet/pods/3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998/volumes"
Jul 30 15:53:06.877136 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.877103    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw8sx\" (UniqueName: \"kubernetes.io/projected/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-kube-api-access-bw8sx\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:53:06.877136 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.877130    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-maas-api-tls\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:53:06.877136 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:06.877140    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8d1fc1-e4a6-4454-bc99-c6e17392349f-openshift-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:53:07.014977 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.014948    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:07.032629 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.032588    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:53:07.435719 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.435694    2582 generic.go:356] "Generic (PLEG): container finished" podID="cc8d1fc1-e4a6-4454-bc99-c6e17392349f" containerID="124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa" exitCode=0
Jul 30 15:53:07.436143 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.435738    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"
Jul 30 15:53:07.436143 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.435776    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z" event={"ID":"cc8d1fc1-e4a6-4454-bc99-c6e17392349f","Type":"ContainerDied","Data":"124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa"}
Jul 30 15:53:07.436143 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.435826    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z" event={"ID":"cc8d1fc1-e4a6-4454-bc99-c6e17392349f","Type":"ContainerDied","Data":"aa022c66072f220fa8ff9c796775c0574dddf05015963a2597c0d23f252d2f48"}
Jul 30 15:53:07.436143 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.435848    2582 scope.go:117] "RemoveContainer" containerID="124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa"
Jul 30 15:53:07.444059 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.444041    2582 scope.go:117] "RemoveContainer" containerID="124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa"
Jul 30 15:53:07.444300 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:53:07.444282    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa\": container with ID starting with 124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa not found: ID does not exist" containerID="124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa"
Jul 30 15:53:07.444420 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.444306    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa"} err="failed to get container status \"124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa\": rpc error: code = NotFound desc = could not find container \"124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa\": container with ID starting with 124b26e8109565d99b089bf70145c121e4968437303817efbdd7f7b8a6b940fa not found: ID does not exist"
Jul 30 15:53:07.455496 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.455475    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"]
Jul 30 15:53:07.458087 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:07.458068    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z"]
Jul 30 15:53:08.806469 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:08.806431    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8d1fc1-e4a6-4454-bc99-c6e17392349f" path="/var/lib/kubelet/pods/cc8d1fc1-e4a6-4454-bc99-c6e17392349f/volumes"
Jul 30 15:53:09.573696 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:09.573656    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-10fa8b-547c6f6489-7f56z" podUID="cc8d1fc1-e4a6-4454-bc99-c6e17392349f" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.49:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 30 15:53:12.056756 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:12.056718    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:18.533370 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:18.533332    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:53:29.452864 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:29.452823    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:29.521066 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:29.521039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:49.052120 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:49.052070    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:49.500558 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:49.500526    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:55.359349 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:55.359300    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:55.459456 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:55.459423    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:55.667180 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:55.667099    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"]
Jul 30 15:53:55.667328 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:55.667306    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy" containerID="cri-o://41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac" gracePeriod=30
Jul 30 15:53:55.703971 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:55.703938    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:55.951559 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:55.951525    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:53:56.058303 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058273    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"]
Jul 30 15:53:56.058592 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058579    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy"
Jul 30 15:53:56.058644 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058594    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy"
Jul 30 15:53:56.058644 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058615    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998" containerName="revoke-keys"
Jul 30 15:53:56.058644 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058623    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998" containerName="revoke-keys"
Jul 30 15:53:56.058644 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058628    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8d1fc1-e4a6-4454-bc99-c6e17392349f" containerName="maas-api"
Jul 30 15:53:56.058644 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058634    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8d1fc1-e4a6-4454-bc99-c6e17392349f" containerName="maas-api"
Jul 30 15:53:56.058794 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058677    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f7d2d3d-2f3c-423e-b3ee-d5f6dbb1b998" containerName="revoke-keys"
Jul 30 15:53:56.058794 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058685    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy"
Jul 30 15:53:56.058794 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058691    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8d1fc1-e4a6-4454-bc99-c6e17392349f" containerName="maas-api"
Jul 30 15:53:56.058794 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.058697    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy"
Jul 30 15:53:56.061719 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.061703    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"
Jul 30 15:53:56.064759 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.064737    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:53:56.077821 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.077791    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"]
Jul 30 15:53:56.165724 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.165693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb\" (UID: \"3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"
Jul 30 15:53:56.266160 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.266086    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb\" (UID: \"3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"
Jul 30 15:53:56.266716 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.266698    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb\" (UID: \"3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"
Jul 30 15:53:56.371418 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.371381    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"
Jul 30 15:53:56.502063 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.502027    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"]
Jul 30 15:53:56.504928 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:53:56.504885    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ecfe6d8_2d8d_429b_868e_4e1dbef15b4b.slice/crio-8a576a34ce8ccb633e16d07d103dc029a9c412e32ee3fb50c524c7d532f7a334 WatchSource:0}: Error finding container 8a576a34ce8ccb633e16d07d103dc029a9c412e32ee3fb50c524c7d532f7a334: Status 404 returned error can't find the container with id 8a576a34ce8ccb633e16d07d103dc029a9c412e32ee3fb50c524c7d532f7a334
Jul 30 15:53:56.658067 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:56.658037    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb" event={"ID":"3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b","Type":"ContainerStarted","Data":"8a576a34ce8ccb633e16d07d103dc029a9c412e32ee3fb50c524c7d532f7a334"}
Jul 30 15:53:57.663907 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:57.663819    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b" containerID="7d0db31cd34e75b274cd68ed1febf8f61e194324a1c3e7f3aec6f39f2d8a48d3" exitCode=0
Jul 30 15:53:57.664274 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:57.663906    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb" event={"ID":"3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b","Type":"ContainerDied","Data":"7d0db31cd34e75b274cd68ed1febf8f61e194324a1c3e7f3aec6f39f2d8a48d3"}
Jul 30 15:53:58.798217 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:58.798192    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"
Jul 30 15:53:58.887499 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:58.887472    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b-maas-api-service-ca\") pod \"3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b\" (UID: \"3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b\") "
Jul 30 15:53:58.887797 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:58.887772    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b" (UID: "3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:53:58.988098 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:58.988074    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:53:59.674361 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:59.674328    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"
Jul 30 15:53:59.674554 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:59.674363    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb" event={"ID":"3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b","Type":"ContainerDied","Data":"8a576a34ce8ccb633e16d07d103dc029a9c412e32ee3fb50c524c7d532f7a334"}
Jul 30 15:53:59.674554 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:53:59.674421    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a576a34ce8ccb633e16d07d103dc029a9c412e32ee3fb50c524c7d532f7a334"
Jul 30 15:54:00.920522 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:00.920493    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:54:01.003730 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.003703    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-certs\") pod \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") "
Jul 30 15:54:01.003911 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.003741    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-socket\") pod \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") "
Jul 30 15:54:01.003911 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.003759    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-data\") pod \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") "
Jul 30 15:54:01.003911 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.003775    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-podinfo\") pod \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") "
Jul 30 15:54:01.003911 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.003795    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-token\") pod \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") "
Jul 30 15:54:01.003911 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.003819    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-envoy\") pod \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") "
Jul 30 15:54:01.003911 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.003842    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-credential-socket\") pod \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") "
Jul 30 15:54:01.003911 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.003904    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istiod-ca-cert\") pod \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") "
Jul 30 15:54:01.004247 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.003965    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ldkr\" (UniqueName: \"kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-kube-api-access-5ldkr\") pod \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\" (UID: \"8cfb7e6e-a088-4ce7-b24d-b6391da26150\") "
Jul 30 15:54:01.004247 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.004014    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8cfb7e6e-a088-4ce7-b24d-b6391da26150" (UID: "8cfb7e6e-a088-4ce7-b24d-b6391da26150"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:01.004247 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.004103    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8cfb7e6e-a088-4ce7-b24d-b6391da26150" (UID: "8cfb7e6e-a088-4ce7-b24d-b6391da26150"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:01.004436 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.004251    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-certs\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:01.004436 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.004235    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8cfb7e6e-a088-4ce7-b24d-b6391da26150" (UID: "8cfb7e6e-a088-4ce7-b24d-b6391da26150"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:01.004436 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.004267    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-workload-socket\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:01.004436 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.004384    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8cfb7e6e-a088-4ce7-b24d-b6391da26150" (UID: "8cfb7e6e-a088-4ce7-b24d-b6391da26150"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:01.004853 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.004767    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-data" (OuterVolumeSpecName: "istio-data") pod "8cfb7e6e-a088-4ce7-b24d-b6391da26150" (UID: "8cfb7e6e-a088-4ce7-b24d-b6391da26150"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:01.006311 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.006283    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-token" (OuterVolumeSpecName: "istio-token") pod "8cfb7e6e-a088-4ce7-b24d-b6391da26150" (UID: "8cfb7e6e-a088-4ce7-b24d-b6391da26150"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:01.006430 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.006311    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-kube-api-access-5ldkr" (OuterVolumeSpecName: "kube-api-access-5ldkr") pod "8cfb7e6e-a088-4ce7-b24d-b6391da26150" (UID: "8cfb7e6e-a088-4ce7-b24d-b6391da26150"). InnerVolumeSpecName "kube-api-access-5ldkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:01.006777 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.006755    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8cfb7e6e-a088-4ce7-b24d-b6391da26150" (UID: "8cfb7e6e-a088-4ce7-b24d-b6391da26150"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:01.006777 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.006762    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8cfb7e6e-a088-4ce7-b24d-b6391da26150" (UID: "8cfb7e6e-a088-4ce7-b24d-b6391da26150"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:54:01.105581 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.105555    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istiod-ca-cert\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:01.105691 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.105582    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ldkr\" (UniqueName: \"kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-kube-api-access-5ldkr\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:01.105691 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.105600    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-data\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:01.105691 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.105612    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-podinfo\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:01.105691 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.105622    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-token\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:01.105691 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.105633    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-istio-envoy\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:01.105691 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.105644    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfb7e6e-a088-4ce7-b24d-b6391da26150-credential-socket\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:01.685766 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.685728    2582 generic.go:356] "Generic (PLEG): container finished" podID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerID="41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac" exitCode=0
Jul 30 15:54:01.685984 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.685776    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"
Jul 30 15:54:01.685984 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.685813    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" event={"ID":"8cfb7e6e-a088-4ce7-b24d-b6391da26150","Type":"ContainerDied","Data":"41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac"}
Jul 30 15:54:01.685984 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.685866    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm" event={"ID":"8cfb7e6e-a088-4ce7-b24d-b6391da26150","Type":"ContainerDied","Data":"264ea6510afd6226d7079ff374ba3fb596f9383e431c2fe94b8e0ed63075c319"}
Jul 30 15:54:01.685984 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.685890    2582 scope.go:117] "RemoveContainer" containerID="41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac"
Jul 30 15:54:01.695331 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.695313    2582 scope.go:117] "RemoveContainer" containerID="921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b"
Jul 30 15:54:01.703176 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.703157    2582 scope.go:117] "RemoveContainer" containerID="41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac"
Jul 30 15:54:01.703438 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:54:01.703419    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac\": container with ID starting with 41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac not found: ID does not exist" containerID="41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac"
Jul 30 15:54:01.703491 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.703448    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac"} err="failed to get container status \"41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac\": rpc error: code = NotFound desc = could not find container \"41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac\": container with ID starting with 41a14f7c36a810180752e38e182eec40a4ae06a249ea8f0d0dc4a53f756cf9ac not found: ID does not exist"
Jul 30 15:54:01.703491 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.703464    2582 scope.go:117] "RemoveContainer" containerID="921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b"
Jul 30 15:54:01.703669 ip-10-0-133-92 kubenswrapper[2582]: E0730 15:54:01.703652    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b\": container with ID starting with 921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b not found: ID does not exist" containerID="921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b"
Jul 30 15:54:01.703714 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.703673    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b"} err="failed to get container status \"921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b\": rpc error: code = NotFound desc = could not find container \"921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b\": container with ID starting with 921165011f356ddfa7f0978fcd100512c895fa7030a684726f27e0432a25601b not found: ID does not exist"
Jul 30 15:54:01.706727 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.706706    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"]
Jul 30 15:54:01.712629 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:01.712610    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-10fa8b-openshift-default-7865b6886b-p9xgm"]
Jul 30 15:54:02.807830 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:02.807796    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" path="/var/lib/kubelet/pods/8cfb7e6e-a088-4ce7-b24d-b6391da26150/volumes"
Jul 30 15:54:06.509457 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.509340    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"]
Jul 30 15:54:06.509892 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.509581    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl" podUID="4a24220e-995e-4476-855c-aee920b4709e" containerName="maas-api" containerID="cri-o://7674a13bcc8032f694f1c4b8283ea554cc48f98d452468284fac16e265b76795" gracePeriod=30
Jul 30 15:54:06.536762 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.536724    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:06.608182 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.608152    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:06.668158 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.668127    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:06.711132 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.711101    2582 generic.go:356] "Generic (PLEG): container finished" podID="4a24220e-995e-4476-855c-aee920b4709e" containerID="7674a13bcc8032f694f1c4b8283ea554cc48f98d452468284fac16e265b76795" exitCode=0
Jul 30 15:54:06.711268 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.711173    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl" event={"ID":"4a24220e-995e-4476-855c-aee920b4709e","Type":"ContainerDied","Data":"7674a13bcc8032f694f1c4b8283ea554cc48f98d452468284fac16e265b76795"}
Jul 30 15:54:06.730448 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.730414    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"]
Jul 30 15:54:06.733680 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.733656    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-0d4073-l2tzb"]
Jul 30 15:54:06.767997 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.767937    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:54:06.806773 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.806739    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b" path="/var/lib/kubelet/pods/3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b/volumes"
Jul 30 15:54:06.850899 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.850872    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-626dg\" (UniqueName: \"kubernetes.io/projected/4a24220e-995e-4476-855c-aee920b4709e-kube-api-access-626dg\") pod \"4a24220e-995e-4476-855c-aee920b4709e\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") "
Jul 30 15:54:06.851005 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.850904    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24220e-995e-4476-855c-aee920b4709e-openshift-service-ca\") pod \"4a24220e-995e-4476-855c-aee920b4709e\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") "
Jul 30 15:54:06.851005 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.850955    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a24220e-995e-4476-855c-aee920b4709e-maas-api-tls\") pod \"4a24220e-995e-4476-855c-aee920b4709e\" (UID: \"4a24220e-995e-4476-855c-aee920b4709e\") "
Jul 30 15:54:06.851315 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.851285    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a24220e-995e-4476-855c-aee920b4709e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4a24220e-995e-4476-855c-aee920b4709e" (UID: "4a24220e-995e-4476-855c-aee920b4709e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:06.853123 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.853097    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a24220e-995e-4476-855c-aee920b4709e-kube-api-access-626dg" (OuterVolumeSpecName: "kube-api-access-626dg") pod "4a24220e-995e-4476-855c-aee920b4709e" (UID: "4a24220e-995e-4476-855c-aee920b4709e"). InnerVolumeSpecName "kube-api-access-626dg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:06.853252 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.853223    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a24220e-995e-4476-855c-aee920b4709e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4a24220e-995e-4476-855c-aee920b4709e" (UID: "4a24220e-995e-4476-855c-aee920b4709e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:54:06.951773 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.951741    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a24220e-995e-4476-855c-aee920b4709e-maas-api-tls\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:06.951773 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.951767    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-626dg\" (UniqueName: \"kubernetes.io/projected/4a24220e-995e-4476-855c-aee920b4709e-kube-api-access-626dg\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:06.951773 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:06.951776    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a24220e-995e-4476-855c-aee920b4709e-openshift-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:07.211241 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:07.211210    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:07.312769 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:07.312743    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:07.717196 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:07.717173    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"
Jul 30 15:54:07.717672 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:07.717212    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl" event={"ID":"4a24220e-995e-4476-855c-aee920b4709e","Type":"ContainerDied","Data":"cd7e67e017b6a6264b5f4a306670f2b690eaeffbc3043232aefb6834bb113b6a"}
Jul 30 15:54:07.717672 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:07.717272    2582 scope.go:117] "RemoveContainer" containerID="7674a13bcc8032f694f1c4b8283ea554cc48f98d452468284fac16e265b76795"
Jul 30 15:54:07.736374 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:07.736350    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"]
Jul 30 15:54:07.743064 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:07.743036    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0d4073-f97dc9486-qfskl"]
Jul 30 15:54:08.812038 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:08.812004    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a24220e-995e-4476-855c-aee920b4709e" path="/var/lib/kubelet/pods/4a24220e-995e-4476-855c-aee920b4709e/volumes"
Jul 30 15:54:09.829011 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:09.828976    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:54:13.305534 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.305505    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:13.371926 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.371895    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:13.556710 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.556620    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"]
Jul 30 15:54:13.556930 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.556858    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy" containerID="cri-o://41d587c418e02aaace44100d9dd09d346da4821b0735f087c8777e5b4d254221" gracePeriod=30
Jul 30 15:54:13.579072 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.579042    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:13.850765 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.850681    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:13.977314 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977280    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"]
Jul 30 15:54:13.977574 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977562    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy"
Jul 30 15:54:13.977623 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977575    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy"
Jul 30 15:54:13.977623 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977585    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a24220e-995e-4476-855c-aee920b4709e" containerName="maas-api"
Jul 30 15:54:13.977623 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977591    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a24220e-995e-4476-855c-aee920b4709e" containerName="maas-api"
Jul 30 15:54:13.977623 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977601    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b" containerName="revoke-keys"
Jul 30 15:54:13.977623 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977607    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b" containerName="revoke-keys"
Jul 30 15:54:13.977623 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977616    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy"
Jul 30 15:54:13.977623 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977621    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy"
Jul 30 15:54:13.977835 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977636    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy"
Jul 30 15:54:13.977835 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977641    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eea024b-6be6-40d3-b693-b483a7da19f7" containerName="istio-proxy"
Jul 30 15:54:13.977835 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977683    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ecfe6d8-2d8d-429b-868e-4e1dbef15b4b" containerName="revoke-keys"
Jul 30 15:54:13.977835 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977692    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a24220e-995e-4476-855c-aee920b4709e" containerName="maas-api"
Jul 30 15:54:13.977835 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977700    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy"
Jul 30 15:54:13.977835 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.977705    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cfb7e6e-a088-4ce7-b24d-b6391da26150" containerName="istio-proxy"
Jul 30 15:54:13.982049 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.982031    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"
Jul 30 15:54:13.984186 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.984163    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:54:13.988632 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:13.988607    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"]
Jul 30 15:54:14.104724 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:14.104630    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab6b3e1f-c272-407d-a59e-3468ac785826-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-845a26-w6htr\" (UID: \"ab6b3e1f-c272-407d-a59e-3468ac785826\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"
Jul 30 15:54:14.205025 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:14.204994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab6b3e1f-c272-407d-a59e-3468ac785826-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-845a26-w6htr\" (UID: \"ab6b3e1f-c272-407d-a59e-3468ac785826\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"
Jul 30 15:54:14.205605 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:14.205587    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab6b3e1f-c272-407d-a59e-3468ac785826-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-845a26-w6htr\" (UID: \"ab6b3e1f-c272-407d-a59e-3468ac785826\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"
Jul 30 15:54:14.293378 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:14.293352    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"
Jul 30 15:54:14.420948 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:14.420924    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"]
Jul 30 15:54:14.423645 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:54:14.423614    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6b3e1f_c272_407d_a59e_3468ac785826.slice/crio-5667ddec630c774945fc81da00925d729ccfc6632465ebfbb7ee0aa197cc6613 WatchSource:0}: Error finding container 5667ddec630c774945fc81da00925d729ccfc6632465ebfbb7ee0aa197cc6613: Status 404 returned error can't find the container with id 5667ddec630c774945fc81da00925d729ccfc6632465ebfbb7ee0aa197cc6613
Jul 30 15:54:14.749820 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:14.749786    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr" event={"ID":"ab6b3e1f-c272-407d-a59e-3468ac785826","Type":"ContainerStarted","Data":"5667ddec630c774945fc81da00925d729ccfc6632465ebfbb7ee0aa197cc6613"}
Jul 30 15:54:15.755553 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:15.755512    2582 generic.go:356] "Generic (PLEG): container finished" podID="ab6b3e1f-c272-407d-a59e-3468ac785826" containerID="421b53ad7aa867ba86b7c24d20a83e9bf299949ae30b41e599e5151ab0fad168" exitCode=0
Jul 30 15:54:15.755966 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:15.755608    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr" event={"ID":"ab6b3e1f-c272-407d-a59e-3468ac785826","Type":"ContainerDied","Data":"421b53ad7aa867ba86b7c24d20a83e9bf299949ae30b41e599e5151ab0fad168"}
Jul 30 15:54:16.890023 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:16.889999    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"
Jul 30 15:54:17.028718 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:17.028643    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab6b3e1f-c272-407d-a59e-3468ac785826-maas-api-service-ca\") pod \"ab6b3e1f-c272-407d-a59e-3468ac785826\" (UID: \"ab6b3e1f-c272-407d-a59e-3468ac785826\") "
Jul 30 15:54:17.029021 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:17.028996    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab6b3e1f-c272-407d-a59e-3468ac785826-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab6b3e1f-c272-407d-a59e-3468ac785826" (UID: "ab6b3e1f-c272-407d-a59e-3468ac785826"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:17.129668 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:17.129645    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab6b3e1f-c272-407d-a59e-3468ac785826-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:17.765658 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:17.765622    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"
Jul 30 15:54:17.765836 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:17.765676    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr" event={"ID":"ab6b3e1f-c272-407d-a59e-3468ac785826","Type":"ContainerDied","Data":"5667ddec630c774945fc81da00925d729ccfc6632465ebfbb7ee0aa197cc6613"}
Jul 30 15:54:17.765836 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:17.765723    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5667ddec630c774945fc81da00925d729ccfc6632465ebfbb7ee0aa197cc6613"
Jul 30 15:54:18.771747 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.771719    2582 generic.go:356] "Generic (PLEG): container finished" podID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerID="41d587c418e02aaace44100d9dd09d346da4821b0735f087c8777e5b4d254221" exitCode=0
Jul 30 15:54:18.772117 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.771746    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" event={"ID":"04a17dbe-4b22-478c-bc66-1d229efdb6fb","Type":"ContainerDied","Data":"41d587c418e02aaace44100d9dd09d346da4821b0735f087c8777e5b4d254221"}
Jul 30 15:54:18.772117 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.771822    2582 scope.go:117] "RemoveContainer" containerID="29bc31b6c898c58232ca9ec1c575adff624c1cb929e9cd0fd75731d7532ee761"
Jul 30 15:54:18.809380 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.809359    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:54:18.944300 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944274    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-socket\") pod \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") "
Jul 30 15:54:18.944511 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944343    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-certs\") pod \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") "
Jul 30 15:54:18.944511 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944369    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-credential-socket\") pod \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") "
Jul 30 15:54:18.944511 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944430    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istiod-ca-cert\") pod \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") "
Jul 30 15:54:18.944511 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944483    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-token\") pod \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") "
Jul 30 15:54:18.944511 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944502    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmkxz\" (UniqueName: \"kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-kube-api-access-rmkxz\") pod \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") "
Jul 30 15:54:18.944775 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944527    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-envoy\") pod \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") "
Jul 30 15:54:18.944775 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944550    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04a17dbe-4b22-478c-bc66-1d229efdb6fb" (UID: "04a17dbe-4b22-478c-bc66-1d229efdb6fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:18.944775 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944560    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-data\") pod \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") "
Jul 30 15:54:18.944775 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944652    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-podinfo\") pod \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\" (UID: \"04a17dbe-4b22-478c-bc66-1d229efdb6fb\") "
Jul 30 15:54:18.945001 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944653    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04a17dbe-4b22-478c-bc66-1d229efdb6fb" (UID: "04a17dbe-4b22-478c-bc66-1d229efdb6fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:18.945001 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944671    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04a17dbe-4b22-478c-bc66-1d229efdb6fb" (UID: "04a17dbe-4b22-478c-bc66-1d229efdb6fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:18.945001 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944842    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04a17dbe-4b22-478c-bc66-1d229efdb6fb" (UID: "04a17dbe-4b22-478c-bc66-1d229efdb6fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:18.945001 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.944906    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-socket\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:18.945459 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.945435    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "04a17dbe-4b22-478c-bc66-1d229efdb6fb" (UID: "04a17dbe-4b22-478c-bc66-1d229efdb6fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:18.946994 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.946924    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04a17dbe-4b22-478c-bc66-1d229efdb6fb" (UID: "04a17dbe-4b22-478c-bc66-1d229efdb6fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 15:54:18.947084 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.947002    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04a17dbe-4b22-478c-bc66-1d229efdb6fb" (UID: "04a17dbe-4b22-478c-bc66-1d229efdb6fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 15:54:18.947281 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.947255    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "04a17dbe-4b22-478c-bc66-1d229efdb6fb" (UID: "04a17dbe-4b22-478c-bc66-1d229efdb6fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:18.947375 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:18.947305    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-kube-api-access-rmkxz" (OuterVolumeSpecName: "kube-api-access-rmkxz") pod "04a17dbe-4b22-478c-bc66-1d229efdb6fb" (UID: "04a17dbe-4b22-478c-bc66-1d229efdb6fb"). InnerVolumeSpecName "kube-api-access-rmkxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:19.046108 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.046069    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-podinfo\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:19.046108 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.046105    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-workload-certs\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:19.046108 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.046115    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-credential-socket\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:19.046108 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.046124    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istiod-ca-cert\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:19.046350 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.046133    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-token\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:19.046350 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.046140    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmkxz\" (UniqueName: \"kubernetes.io/projected/04a17dbe-4b22-478c-bc66-1d229efdb6fb-kube-api-access-rmkxz\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:19.046350 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.046149    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-envoy\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:19.046350 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.046156    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04a17dbe-4b22-478c-bc66-1d229efdb6fb-istio-data\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:19.782673 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.782636    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6" event={"ID":"04a17dbe-4b22-478c-bc66-1d229efdb6fb","Type":"ContainerDied","Data":"5d035add9b6d3e51e084d32561ffc8592c4e333d8fdc5540eaf348fae43d6459"}
Jul 30 15:54:19.782673 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.782670    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"
Jul 30 15:54:19.783188 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.782729    2582 scope.go:117] "RemoveContainer" containerID="41d587c418e02aaace44100d9dd09d346da4821b0735f087c8777e5b4d254221"
Jul 30 15:54:19.804248 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.804216    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"]
Jul 30 15:54:19.810521 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:19.810498    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-0d4073-openshift-default-65bd7fc488-hmzx6"]
Jul 30 15:54:20.806968 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:20.806934    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" path="/var/lib/kubelet/pods/04a17dbe-4b22-478c-bc66-1d229efdb6fb/volumes"
Jul 30 15:54:24.619043 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:24.619009    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:24.622958 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:24.622934    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z"]
Jul 30 15:54:24.626764 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:24.626745    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-845a26-29757105-crp5z"]
Jul 30 15:54:24.686640 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:24.686610    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:24.742232 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:24.742200    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:24.807030 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:24.806996    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="424ab002-57a5-414a-a3bc-3eb5d6681ef4" path="/var/lib/kubelet/pods/424ab002-57a5-414a-a3bc-3eb5d6681ef4/volumes"
Jul 30 15:54:24.813604 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:24.813577    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"]
Jul 30 15:54:24.815845 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:24.815825    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-845a26-w6htr"]
Jul 30 15:54:24.817355 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:24.817333    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:25.207952 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:25.207909    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:26.806829 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:26.806800    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab6b3e1f-c272-407d-a59e-3468ac785826" path="/var/lib/kubelet/pods/ab6b3e1f-c272-407d-a59e-3468ac785826/volumes"
Jul 30 15:54:27.728818 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:27.728785    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:54:31.632677 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:31.632645    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:31.733976 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:31.733946    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:31.901361 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:31.901266    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:32.175637 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.175540    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:32.281371 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.281337    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"]
Jul 30 15:54:32.281642 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.281629    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy"
Jul 30 15:54:32.281692 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.281643    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy"
Jul 30 15:54:32.281692 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.281656    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab6b3e1f-c272-407d-a59e-3468ac785826" containerName="revoke-keys"
Jul 30 15:54:32.281692 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.281662    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab6b3e1f-c272-407d-a59e-3468ac785826" containerName="revoke-keys"
Jul 30 15:54:32.281790 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.281715    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy"
Jul 30 15:54:32.281790 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.281723    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy"
Jul 30 15:54:32.281790 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.281730    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab6b3e1f-c272-407d-a59e-3468ac785826" containerName="revoke-keys"
Jul 30 15:54:32.284585 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.284569    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"
Jul 30 15:54:32.286860 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.286843    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rmv59\""
Jul 30 15:54:32.293092 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.293070    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"]
Jul 30 15:54:32.444167 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.444140    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23568710-905a-4c44-9043-70e9eb733bec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z\" (UID: \"23568710-905a-4c44-9043-70e9eb733bec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"
Jul 30 15:54:32.544873 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.544836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23568710-905a-4c44-9043-70e9eb733bec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z\" (UID: \"23568710-905a-4c44-9043-70e9eb733bec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"
Jul 30 15:54:32.545431 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.545387    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23568710-905a-4c44-9043-70e9eb733bec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z\" (UID: \"23568710-905a-4c44-9043-70e9eb733bec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"
Jul 30 15:54:32.595550 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.595518    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"
Jul 30 15:54:32.727750 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.727552    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"]
Jul 30 15:54:32.730560 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:54:32.730536    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23568710_905a_4c44_9043_70e9eb733bec.slice/crio-fe63176e730c9d256d218a77ff86019937cc7fbb602834356639e55c989e6bcb WatchSource:0}: Error finding container fe63176e730c9d256d218a77ff86019937cc7fbb602834356639e55c989e6bcb: Status 404 returned error can't find the container with id fe63176e730c9d256d218a77ff86019937cc7fbb602834356639e55c989e6bcb
Jul 30 15:54:32.840599 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:32.840570    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z" event={"ID":"23568710-905a-4c44-9043-70e9eb733bec","Type":"ContainerStarted","Data":"fe63176e730c9d256d218a77ff86019937cc7fbb602834356639e55c989e6bcb"}
Jul 30 15:54:33.846214 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:33.846118    2582 generic.go:356] "Generic (PLEG): container finished" podID="23568710-905a-4c44-9043-70e9eb733bec" containerID="a768cae5ef3bca61fb152651058a82f96774184746992a153e85a046f0f2811c" exitCode=0
Jul 30 15:54:33.846214 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:33.846198    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z" event={"ID":"23568710-905a-4c44-9043-70e9eb733bec","Type":"ContainerDied","Data":"a768cae5ef3bca61fb152651058a82f96774184746992a153e85a046f0f2811c"}
Jul 30 15:54:34.982583 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:34.982555    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"
Jul 30 15:54:35.065564 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:35.065542    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23568710-905a-4c44-9043-70e9eb733bec-maas-api-service-ca\") pod \"23568710-905a-4c44-9043-70e9eb733bec\" (UID: \"23568710-905a-4c44-9043-70e9eb733bec\") "
Jul 30 15:54:35.065848 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:35.065826    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23568710-905a-4c44-9043-70e9eb733bec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "23568710-905a-4c44-9043-70e9eb733bec" (UID: "23568710-905a-4c44-9043-70e9eb733bec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:35.166595 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:35.166529    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23568710-905a-4c44-9043-70e9eb733bec-maas-api-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:35.855717 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:35.855690    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"
Jul 30 15:54:35.855946 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:35.855696    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z" event={"ID":"23568710-905a-4c44-9043-70e9eb733bec","Type":"ContainerDied","Data":"fe63176e730c9d256d218a77ff86019937cc7fbb602834356639e55c989e6bcb"}
Jul 30 15:54:35.855946 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:35.855816    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe63176e730c9d256d218a77ff86019937cc7fbb602834356639e55c989e6bcb"
Jul 30 15:54:42.734217 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.734181    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"]
Jul 30 15:54:42.734738 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.734375    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv" podUID="a04052c3-5a6f-4312-bc9b-a8fc05900cc8" containerName="maas-api" containerID="cri-o://a7f50c4f114c1ac5de5d6adb05a05232b99b3d2a3e83a79aa4fc58413b495cca" gracePeriod=30
Jul 30 15:54:42.763151 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.763120    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22"]
Jul 30 15:54:42.765185 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.765156    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2b1f1f-29757105-cgh22"]
Jul 30 15:54:42.772524 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.772497    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:42.809689 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.809654    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="464dfab2-2ee7-42c4-b47f-59a83519dc95" path="/var/lib/kubelet/pods/464dfab2-2ee7-42c4-b47f-59a83519dc95/volumes"
Jul 30 15:54:42.847347 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.847317    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:42.887701 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.887673    2582 generic.go:356] "Generic (PLEG): container finished" podID="a04052c3-5a6f-4312-bc9b-a8fc05900cc8" containerID="a7f50c4f114c1ac5de5d6adb05a05232b99b3d2a3e83a79aa4fc58413b495cca" exitCode=0
Jul 30 15:54:42.887868 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.887723    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv" event={"ID":"a04052c3-5a6f-4312-bc9b-a8fc05900cc8","Type":"ContainerDied","Data":"a7f50c4f114c1ac5de5d6adb05a05232b99b3d2a3e83a79aa4fc58413b495cca"}
Jul 30 15:54:42.947572 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.947543    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:42.953160 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.953134    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"]
Jul 30 15:54:42.955268 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.955245    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2b1f1f-kh65z"]
Jul 30 15:54:42.985745 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:42.985684    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:54:43.047739 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.047692    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:43.132996 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.132966    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-openshift-service-ca\") pod \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") "
Jul 30 15:54:43.133167 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.133039    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-maas-api-tls\") pod \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") "
Jul 30 15:54:43.133167 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.133087    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkghz\" (UniqueName: \"kubernetes.io/projected/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-kube-api-access-bkghz\") pod \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\" (UID: \"a04052c3-5a6f-4312-bc9b-a8fc05900cc8\") "
Jul 30 15:54:43.133371 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.133352    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a04052c3-5a6f-4312-bc9b-a8fc05900cc8" (UID: "a04052c3-5a6f-4312-bc9b-a8fc05900cc8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 15:54:43.135235 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.135206    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-kube-api-access-bkghz" (OuterVolumeSpecName: "kube-api-access-bkghz") pod "a04052c3-5a6f-4312-bc9b-a8fc05900cc8" (UID: "a04052c3-5a6f-4312-bc9b-a8fc05900cc8"). InnerVolumeSpecName "kube-api-access-bkghz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 15:54:43.135359 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.135239    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a04052c3-5a6f-4312-bc9b-a8fc05900cc8" (UID: "a04052c3-5a6f-4312-bc9b-a8fc05900cc8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 15:54:43.233868 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.233842    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-maas-api-tls\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:43.233868 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.233866    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkghz\" (UniqueName: \"kubernetes.io/projected/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-kube-api-access-bkghz\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:43.233868 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.233876    2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a04052c3-5a6f-4312-bc9b-a8fc05900cc8-openshift-service-ca\") on node \"ip-10-0-133-92.ec2.internal\" DevicePath \"\""
Jul 30 15:54:43.408747 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.408663    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:43.893088 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.893059    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"
Jul 30 15:54:43.893088 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.893079    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv" event={"ID":"a04052c3-5a6f-4312-bc9b-a8fc05900cc8","Type":"ContainerDied","Data":"322e8aa499e34d751efe20ecdfd53bfe8e94feac176b643db513dbd06d60fbe0"}
Jul 30 15:54:43.893651 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.893140    2582 scope.go:117] "RemoveContainer" containerID="a7f50c4f114c1ac5de5d6adb05a05232b99b3d2a3e83a79aa4fc58413b495cca"
Jul 30 15:54:43.914452 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.914425    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"]
Jul 30 15:54:43.918250 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:43.918227    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2b1f1f-9d7556844-9cvhv"]
Jul 30 15:54:44.807200 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:44.807164    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23568710-905a-4c44-9043-70e9eb733bec" path="/var/lib/kubelet/pods/23568710-905a-4c44-9043-70e9eb733bec/volumes"
Jul 30 15:54:44.807624 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:44.807604    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a04052c3-5a6f-4312-bc9b-a8fc05900cc8" path="/var/lib/kubelet/pods/a04052c3-5a6f-4312-bc9b-a8fc05900cc8/volumes"
Jul 30 15:54:45.933696 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:45.933661    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nrsng"]
Jul 30 15:54:49.748088 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:49.748055    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:49.848868 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:49.848815    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:50.062326 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:50.062231    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:50.291466 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:50.291435    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:54:56.817248 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:56.817215    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-76v7f_6c0b168d-4cad-4641-8c8d-d92f50c708b2/server/0.log"
Jul 30 15:54:57.196712 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:57.196689    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb_30327971-ddc9-410e-8573-5ddd8ac0945a/copy-manifests/0.log"
Jul 30 15:54:57.202841 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:57.202819    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb_30327971-ddc9-410e-8573-5ddd8ac0945a/manager/0.log"
Jul 30 15:54:57.941314 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:57.941284    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757090-fw7ln_dd670b79-4237-4090-b43d-76df18264d7d/cleanup/0.log"
Jul 30 15:54:58.055862 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:54:58.055836    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757105-74882_b4fa00f2-3a30-4925-8564-1d1c02d74e84/cleanup/0.log"
Jul 30 15:55:00.328149 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:00.328122    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nrsng_b1d79f03-bdb7-4a7d-9261-232438552ada/limitador/0.log"
Jul 30 15:55:00.969038 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:00.969005    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-s4qms_8d69921a-091f-4898-98be-df24e8bb429f/discovery/0.log"
Jul 30 15:55:01.709997 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:01.709968    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76489d7779-qr8pc_f31dcb58-4fd7-425b-b5a8-51dfac54437f/router/0.log"
Jul 30 15:55:02.720435 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:02.720377    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-jfhnt_cb107ec1-19ae-4acb-9e2f-ff6a58277c9b/main/0.log"
Jul 30 15:55:08.347699 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:08.347665    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:55:08.428173 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:08.428142    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log" limit=1024
Jul 30 15:55:11.389702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:11.389666    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n8tbb_478b6274-9630-42e8-b363-e3e088f1bdbb/global-pull-secret-syncer/0.log"
Jul 30 15:55:11.484744 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:11.484711    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5gsdl_c00a8c8a-3d2a-48d2-873e-b928edb51756/konnectivity-agent/0.log"
Jul 30 15:55:11.635530 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:11.635497    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-92.ec2.internal_958f474d6621921ad2d3c1a4ebe5848f/haproxy/0.log"
Jul 30 15:55:15.193935 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:15.193908    2582 scope.go:117] "RemoveContainer" containerID="58a1f07b83267a1f0c386183c550598ae1d23076a9cf94e909a7ec41585c3996"
Jul 30 15:55:15.201638 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:15.201619    2582 scope.go:117] "RemoveContainer" containerID="8c8d0902a8643c38514fcb47123086c63e006805e85057ad1578b4c3625ff707"
Jul 30 15:55:17.324512 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:17.324481    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nrsng_b1d79f03-bdb7-4a7d-9261-232438552ada/limitador/0.log"
Jul 30 15:55:19.577636 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:19.577608    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ccbwd_e53f9fde-be7d-43e1-98c2-48bf0148b3e4/node-exporter/0.log"
Jul 30 15:55:19.601762 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:19.601735    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ccbwd_e53f9fde-be7d-43e1-98c2-48bf0148b3e4/kube-rbac-proxy/0.log"
Jul 30 15:55:19.628967 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:19.628930    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ccbwd_e53f9fde-be7d-43e1-98c2-48bf0148b3e4/init-textfile/0.log"
Jul 30 15:55:21.597987 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.597954    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-jbpq8_f95cd5c5-f2a4-4e2d-bd6b-5e75bea9abce/networking-console-plugin/0.log"
Jul 30 15:55:21.769101 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.769063    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"]
Jul 30 15:55:21.769349 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.769336    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy"
Jul 30 15:55:21.769421 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.769350    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a17dbe-4b22-478c-bc66-1d229efdb6fb" containerName="istio-proxy"
Jul 30 15:55:21.769421 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.769364    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a04052c3-5a6f-4312-bc9b-a8fc05900cc8" containerName="maas-api"
Jul 30 15:55:21.769421 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.769369    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a04052c3-5a6f-4312-bc9b-a8fc05900cc8" containerName="maas-api"
Jul 30 15:55:21.769421 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.769384    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23568710-905a-4c44-9043-70e9eb733bec" containerName="revoke-keys"
Jul 30 15:55:21.769421 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.769390    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="23568710-905a-4c44-9043-70e9eb733bec" containerName="revoke-keys"
Jul 30 15:55:21.769581 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.769452    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a04052c3-5a6f-4312-bc9b-a8fc05900cc8" containerName="maas-api"
Jul 30 15:55:21.769581 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.769459    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="23568710-905a-4c44-9043-70e9eb733bec" containerName="revoke-keys"
Jul 30 15:55:21.773925 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.773909    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.776490 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.776468    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ww6sr\"/\"default-dockercfg-sp7jx\""
Jul 30 15:55:21.776616 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.776468    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ww6sr\"/\"kube-root-ca.crt\""
Jul 30 15:55:21.776784 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.776769    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ww6sr\"/\"openshift-service-ca.crt\""
Jul 30 15:55:21.781225 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.781205    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"]
Jul 30 15:55:21.829466 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.829439    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-sys\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.829466 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.829467    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-podres\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.829718 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.829485    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-lib-modules\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.829718 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.829504    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-proc\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.829718 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.829586    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdvrb\" (UniqueName: \"kubernetes.io/projected/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-kube-api-access-jdvrb\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.930833 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.930729    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-sys\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.930833 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.930769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-podres\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.930833 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.930791    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-lib-modules\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.930833 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.930821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-proc\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.931194 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.930876    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-sys\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.931194 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.930903    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-podres\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.931194 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.930883    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-proc\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.931194 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.930931    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-lib-modules\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.931194 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.930956    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdvrb\" (UniqueName: \"kubernetes.io/projected/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-kube-api-access-jdvrb\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:21.940270 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:21.940241    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdvrb\" (UniqueName: \"kubernetes.io/projected/11ba1ec9-6a98-4faa-9dd2-bef86eebfef1-kube-api-access-jdvrb\") pod \"perf-node-gather-daemonset-mbhps\" (UID: \"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1\") " pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:22.084645 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:22.084615    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:22.214901 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:22.214860    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"]
Jul 30 15:55:22.218068 ip-10-0-133-92 kubenswrapper[2582]: W0730 15:55:22.218042    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod11ba1ec9_6a98_4faa_9dd2_bef86eebfef1.slice/crio-91a6d21904b8bd4f81a4965121ec065bb21f1ea8f2839742248c74de6001d07c WatchSource:0}: Error finding container 91a6d21904b8bd4f81a4965121ec065bb21f1ea8f2839742248c74de6001d07c: Status 404 returned error can't find the container with id 91a6d21904b8bd4f81a4965121ec065bb21f1ea8f2839742248c74de6001d07c
Jul 30 15:55:23.079021 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:23.076341    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps" event={"ID":"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1","Type":"ContainerStarted","Data":"a4f95d17caee60dabea777fccc1db9d17635f17dbaab90093755ce2a232e1983"}
Jul 30 15:55:23.079021 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:23.076427    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps" event={"ID":"11ba1ec9-6a98-4faa-9dd2-bef86eebfef1","Type":"ContainerStarted","Data":"91a6d21904b8bd4f81a4965121ec065bb21f1ea8f2839742248c74de6001d07c"}
Jul 30 15:55:23.079021 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:23.076542    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:23.096366 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:23.096321    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps" podStartSLOduration=2.096304993 podStartE2EDuration="2.096304993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 15:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 15:55:23.095019809 +0000 UTC m=+4028.768465921" watchObservedRunningTime="2026-07-30 15:55:23.096304993 +0000 UTC m=+4028.769751105"
Jul 30 15:55:23.472906 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:23.472883    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-8z9mx_bea29110-940e-491a-844b-966b31f8dea2/volume-data-source-validator/0.log"
Jul 30 15:55:24.592304 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:24.592275    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jjcpx_e7cd0a49-d9f2-42d7-9f3a-abc723a0d944/dns/0.log"
Jul 30 15:55:24.616963 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:24.616930    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jjcpx_e7cd0a49-d9f2-42d7-9f3a-abc723a0d944/kube-rbac-proxy/0.log"
Jul 30 15:55:24.748597 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:24.748555    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vhmks_19783e54-8469-4c0d-906d-f30863b87ea6/dns-node-resolver/0.log"
Jul 30 15:55:25.423248 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:25.423216    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x2w25_64206ee8-2cf7-42dd-9771-4ee52fd66a97/node-ca/0.log"
Jul 30 15:55:26.593698 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:26.593667    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-s4qms_8d69921a-091f-4898-98be-df24e8bb429f/discovery/0.log"
Jul 30 15:55:26.808407 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:26.808368    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76489d7779-qr8pc_f31dcb58-4fd7-425b-b5a8-51dfac54437f/router/0.log"
Jul 30 15:55:27.725205 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:27.725170    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rzcsk_cb36fd80-2c52-40fd-b9bf-4868a44b4030/serve-healthcheck-canary/0.log"
Jul 30 15:55:28.236661 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:28.236631    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2dlwr_18005be7-0022-4919-81ff-9cc06fc5aa5b/kube-rbac-proxy/0.log"
Jul 30 15:55:28.260447 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:28.260420    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2dlwr_18005be7-0022-4919-81ff-9cc06fc5aa5b/exporter/0.log"
Jul 30 15:55:28.282714 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:28.282686    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2dlwr_18005be7-0022-4919-81ff-9cc06fc5aa5b/extractor/0.log"
Jul 30 15:55:29.090347 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:29.090321    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ww6sr/perf-node-gather-daemonset-mbhps"
Jul 30 15:55:31.329686 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:31.329660    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-76v7f_6c0b168d-4cad-4641-8c8d-d92f50c708b2/server/0.log"
Jul 30 15:55:31.573702 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:31.573633    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb_30327971-ddc9-410e-8573-5ddd8ac0945a/manager/0.log"
Jul 30 15:55:31.596707 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:31.596687    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bdff7d7c7-lnpmb_30327971-ddc9-410e-8573-5ddd8ac0945a/copy-manifests/0.log"
Jul 30 15:55:32.083367 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:32.083337    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-s4qms_8d69921a-091f-4898-98be-df24e8bb429f/discovery/0.log"
Jul 30 15:55:32.227805 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:32.227781    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76489d7779-qr8pc_f31dcb58-4fd7-425b-b5a8-51dfac54437f/router/0.log"
Jul 30 15:55:32.915650 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:32.915617    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b696b7c-fj29g_add9f030-6c98-4832-a445-402a4926bdfb/manager/0.log"
Jul 30 15:55:33.866337 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:33.866310    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-s4qms_8d69921a-091f-4898-98be-df24e8bb429f/discovery/0.log"
Jul 30 15:55:34.017844 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:34.017814    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76489d7779-qr8pc_f31dcb58-4fd7-425b-b5a8-51dfac54437f/router/0.log"
Jul 30 15:55:38.282312 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:38.282278    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql_2b9a1df1-f0ea-4efb-b71d-ab8a20761180/kube-storage-version-migrator-operator/1.log"
Jul 30 15:55:38.283051 ip-10-0-133-92 kubenswrapper[2582]: I0730 15:55:38.283029    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-nxvql_2b9a1df1-f0ea-4efb-b71d-ab8a20761180/kube-storage-version-migrator-operator/0.log"